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Abstract 

Over  the  past  five  to  seven  years,  the  United  States  Air  Force  has  begun  to 
employ  online  Communities  of  Practice  (CoP)  as  a  means  to  collaborate  virtually. 

During  this  time,  there  have  been  several  studies  of  these  online  communities  in  hopes  to 
better  understand  their  use,  as  well  as  lack  of  use.  The  primary  goal  of  this  research  is  to 
apply  the  theories  of  Davis’  (1989)  technology  acceptance  model  to  identify  the  factors 
that  affect  the  acceptance  and  use  of  CoPs.  These  findings  would  then  be  used  to  provide 
suggestions  on  how  to  improve  the  acceptance  and  use  of  CoPs  to  CoP  administrators  and 
ultimately  to  Air  Force  Knowledge  Now  (AFKN),  the  managerial  owners  of  all  CoPs. 

This  research  used  a  mixed  method  strategy  to  collect  data,  which  incorporated 
data  from  a  previous  research  study  on  AFKN  CoPs,  a  pre-interview  survey,  and  an 
interview  that  included  both  open  and  closed  ended  questions.  This  method  allowed  the 
researcher  to  converge  on  the  broad  results  in  order  to  focus  on  detailed  views  from  the 
participants.  (Creswell,  2003) 

The  findings  from  this  research  suggest  differences  in  perceptions  of  users  based 
on  functional  makeup,  formality,  access,  length  of  use,  or  user’s  grade.  Additionally,  the 
factors  of  social  influence,  facilitating  conditions,  and  user  acceptance  enablers  strongly 
influenced  the  usage  behavior  of  CoP  users.  Finally,  the  interview  process  exposed 
numerous  factors  that  encouraged  and  discouraged  use  of  the  CoPs.  These  findings  were 
presented  as  recommendations  for  both  AFKN  and  CoP  administrators  to  help  improve 
the  quality  of  CoPs. 
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TECHNOLOGY  ACCEPTANCE  AND  USE  IN  A  KNOWLEDGE 


MANAGEMENT  SUPPORT  SYSTEM:  AN  EXPLORATORY  CASE  STUDY  OF 

AIR  FORCE  KNOWLEDGE  NOW  COMMUNITIES  OF  PRACTICE 

I.  Introduction 

Background 

Goal  number  seven  of  the  U.S.  Air  Force’s  Strategic  Direction  states: 

“Implement  knowledge  management  (KM)  practices  and  technologies  to  assure 
knowledge  is  identified,  captured,  and  shared.”  (USAF,  2004)  One  way  in  which  the 
USAF  is  pursuing  this  goal  is  through  the  Community  of  Practice  (CoP).  The  Air  Force’s 
Chief  Information  Officer  (CIO)  has  tasked  Air  Force  Knowledge  Now  (AFKN)  to  host 
these  online  CoPs  as  a  means  to  enhance  and  facilitate  KM.  (AF/CIO,  2002)  These 
CoPs  are  intended  to  provide  users,  which  share  a  functional  or  organizational  bond,  the 
ability  to  electronically  collaborate.  There  have  been  several  studies  on  CoPs  have  been 
previously  performed  at  the  Air  Force  Institute  of  Technology  (AFIT).  The  current 
research  will  explore  some  of  the  findings  from  these  previous  studies,  while  applying  the 
theories  of  the  Technology  Acceptance  Model  (TAM)  in  order  to  gain  a  better 
understanding  of  the  use  and  acceptance  of  CoPs. 

Problem  Statement 

As  of  November  2004,  there  are  681  active  CoPs  and  280  inactive  CoPs,  which 
equates  to  a  failure  rate  of  around  30%.  As  stated  above,  there  have  been  several  studies 
performed  at  AFIT  looking  at  the  many  theories  of  factors  that  help  “cultivate”  new 
CoPs.  Although  these  studies  have  looked  at  many  facets  of  CoPs,  some  element  still 
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seems  to  be  missing;  this  30%  failure  rate  highlights  this  point.  The  current  research  is 
focused  on  CoPs  as  a  form  of  technology,  and  as  such,  it  will  study  CoP  acceptance  and 
use  from  a  similar  perspective  to  any  other  information  technology  (IT)  system.  There 
have  been  numerous  studies  based  on  Davis’  1989  work  on  TAM  (Davis,  1989);  but  to 
date  Knowledge  Management  Systems  (KMS)  such  as  the  AFKN  CoPs  have  been 
relatively  unexplored.  (Dasgupta,  Granger,  &  McGarry,  2002)  This  study  embraces  the 
recommendation  of  Venkatesh  et  ah,  to  research  technologies  such  as  collaborative 
systems  in  order  to  provide  a  “richer  understanding  of  technology  adoption  and  usage 
behavior.”  (Venkatesh,  Morris,  Davis,  &  Davis,  2003) 

Research  Questions 

This  research  seeks  to  discover  if  there  are  a  specific  set  of  factors  that  CoP  or 
AFKN  administrators  can  incorporate  into  CoPs  to  encourage  acceptance  and  use.  These 
factors  may  or  may  not  be  affected  based  on  CoP  functional  makeup,  formality,  access, 
length  of  use  or  user’s  grade. 

Investigative  Questions 

la.  Based  on  existing  models,  is  there  a  difference  between  factors  based  on 
whether  the  CoP  is  used  by  teams,  function,  or  directorates? 

lb.  Based  on  existing  models,  is  there  a  difference  between  factors  based  on 
whether  the  CoP  is  formed  informally  or  formally? 

lc.  Based  on  existing  models,  is  there  a  difference  between  factors  based  on 
whether  the  CoP  is  open  or  closed? 

ld.  Based  on  existing  models,  is  there  a  difference  between  factors  based  on  how 
long  the  individual  has  been  with  the  CoP? 

le.  Based  on  existing  models,  is  there  a  difference  between  factors  based  on  the 
individual’s  grade/position? 
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2a.  What  are  the  specific  factors  that  encouraged  an  individual  to  participate  in  a 
particular  CoP  when  initially  starting  to  use  the  CoP? 

2b.  What  factors  discouraged  an  individual  from  participating  in  a  particular  CoP 
when  initially  starting  to  use  the  CoP? 

3a.  What  factors  encouraged  an  individual  to  participate  in  a  particular  CoP  after 
initial  use? 

3b.  What  factors  discouraged  an  individual  from  participating  in  a  particular  CoP 
after  initial  use? 

4.  How  are  CoP  users  using  CoPs? 

Research  Focus 

Although  there  are  many  instances  of  CoPs  that  are  being  used  in  the  civilian 
sector,  the  focus  of  this  research  will  be  on  AFKN  CoPs,  with  the  individual  CoP  user 
being  the  unit  of  analysis.  Additionally,  other  factors  such  as  access,  formality,  time 
using  the  CoP,  position,  and/or  organizational  composition  will  be  looked  at  to  identify 
potential  explanations  for  differences.  The  underlying  focus  of  this  research  will  seek  to 
find  out  how  AFKN  CoPs  are  being  used. 

Methodology 

To  answer  investigative  question  one,  a  quantitative  analysis  of  data  collected 
during  a  previous  AFKN  CoP  study  (Fitzgerald,  2004;  Hinrichsen,  2004)  will  be 
performed.  To  answer  investigative  questions  two  and  three,  a  case  study  will  be 
performed  to  identify  specific  factors  that  may  or  may  not  affect  usage  and  acceptance  of 
KMS.  The  model  for  this  case  study  will  be  constructed  based  on  findings  of  a 
comprehensive  literature  review.  These  findings  will  also  be  used  to  construct  the  survey 
and  interview  instrument  that  will  be  used  during  the  case  study  portion  of  this  research. 
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The  case  study  will  encompass  one-on-one  telephone  interviews,  with  CoP  users.  The 
results  from  the  interviews  will  be  examined  to  answer  investigative  question  four. 

Scope 

The  scope  of  this  research  effort  will  explore  the  factors  affecting  acceptance  and 
use  within  AFKN  CoPs.  To  do  this,  the  research  will  review  existing  literature  to 
identify  factors  affecting  participation  in  other  forms  of  computer-mediated 
communication  (ex.  group  support  systems);  with  the  goal  of  identifying  the  essential 
factors  involved  in  successful  participation  within  collaborative  knowledge  management 
systems  such  as  CoPs.  The  results  will  potentially  be  used  to  aid  in  the  modification  and 
management  of  existing  AFKN  CoPs,  as  well  as  in  the  design  and  implementation  of 
future  CoPs. 

Limitations 

Limitations  of  this  research  include  the  small  sample  of  the  population  of  AFKN 
CoP  users/administrators,  due  to  the  nature  of  a  case  study.  Additionally,  as  stated  earlier 
this  study  is  only  looking  at  the  AFKN  CoPs  and  therefore  the  results  of  this  study  may 
not  be  transferable  to  other  KMS  or  information  technology  (IT)  acceptance  in  general. 

Chapter  Summary 

This  chapter  reviewed  the  background  on  CoPs  as  well  as  the  Air  Force’s  current 
initiatives  to  implement  the  use  of  CoPs.  Additionally,  the  theories  of  TAM  were 
identified  as  the  underlying  construct  of  this  research.  The  problem  statement  concerning 
use  and  acceptance  of  CoPs  and  a  general  overview  of  the  methodology  that  will  be  used 
was  also  addressed.  Furthermore,  this  chapter  discussed  advantages  that  the  research 
may  provide  for  the  Air  Force  as  well  as  some  of  the  limitations  of  the  research. 
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Next,  a  literature  review  will  be  presented  in  chapter  2.  The  scope  of  the 
literature  review  includes  the  thinking  of  experts  and  academics  from  peer-reviewed 
journal  articles  and  books  as  it  applies  to  this  research  as  well  as  a  thorough  review  of 
several  previously  performed  studies  of  AFKN  CoPs.  After  the  literature  review,  chapter 
three  will  discuss  the  specific  research  methodology  that  will  be  used  to  conduct  the 
study.  Chapter  four  will  provide  the  results  of  the  research  and  analysis  of  the  data. 
Lastly,  chapter  five  will  discuss  the  implications  of  the  research,  some  suggested  uses  of 
the  implications,  and  some  possibilities  for  future  research. 
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II.  Literature  Review 


Overview 

This  thesis  research  attempts  to  identify  factors  affecting  use  and  acceptance  of 
AFKN  CoPs  based  on  the  theories  of  the  TAM.  The  scope  of  this  literature  review 
represents  the  thinking  of  experts  and  academics  from  numerous  journal  articles  and 
books  pertaining  to  technology  acceptance  and  use  of  IT  and  KM  systems.  The 
information  in  this  literature  review  defines  what  CoPs  are  and  describes  some  of  the 
factors  that  affect  knowledge  transfer  and  acceptance  of  this  technology.  The 
information  within  this  chapter  will  be  presented  in  three  parts:  defining  CoPs  and  their 
uses,  review  of  previous  AFIT  studies  of  AFKN  CoPs,  and  finally  a  review  of  literature 
in  regards  to  technology  acceptance.  The  chapter  will  conclude  with  a  comprehensive 
description  of  the  research  model  that  will  be  used  to  address  this  research. 
Communities  of  Practice 

Up  to  this  point,  CoPs  have  not  been  formally  defined.  Wenger  (2002)  defines  a 
Community  of  Practice  as  a  group  of  people  “who  share  a  concern,  set  of  problems,  or  a 
passion  about  a  topic  and  who  deepen  their  knowledge  and  expertise  in  this  area  by 
interacting  on  an  ongoing  basis.”  (Wenger,  2002)  Although  this  research  refers  to  the 
AFKN  CoP  as  a  CoP,  AFKN  recognizes  that  their  CoPs  are  actually  just  “workspaces” 
for  CoPs  that  provide: 

“...a  web-based  collaborative  environment  where  members  of  a  group  use 
shared  information  and  administrative  and  communications  tools  to  conduct 
business,  manage  a  project,  keep  abreast  of  important  group  issues  and  solve 
group  problems.”  (AFKN,  2004) 
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Wenger  makes  it  a  point  to  show  a  distinction  between  what  he  defines  as  a  CoP 
as  opposed  to  formal  departments,  operational  teams,  project  teams,  communities  of 
interest,  or  informal  networks.  One  of  the  key  differences  between  a  CoP  and  any  of  the 
other  structures  is  the  purpose.  A  CoP’s  purpose  is  “to  create  and  exchange  knowledge 
and  to  develop  individual  capabilities.”  The  purpose  of  the  other  structures  include: 
delivering  a  product  or  service,  taking  care  of  an  ongoing  operation  or  process, 
accomplishing  a  specific  task,  infonning  a  group  (a  form  of  electronic  bulletin  board),  or 
informally  receiving  and  passing  on  information.  (Wenger,  2002) 

Based  on  the  previous  two  paragraphs,  the  AFKN  CoPs  can  be  regarded  as  an  IT 
front-end  that  could  be  used  for  the  majority  of  the  other  structures  that  were  identified 
by  Wenger  and  not  just  exclusively  a  CoP.  Although  previous  AFKN  research  defines  a 
CoP  using  Wenger’s  (2002)  definition  of  a  CoP,  the  current  research  defines  the  AFKN 
CoPs  as  a  graphical  interface,  more  specifically  defined  as  a  knowledge  management 
support  system  (KMSS),  which  “facilitate  access  to  and  retrieval  of  content.”  (Alavi  & 
Leidner,  2001) 

Previous  Air  Force  Knowledge  Now  Research 

Bartczak  (2002)  performed  one  of  the  first  studies  of  the  AFKN  CoPs  as  part  of  a 
PhD  dissertation.  The  purpose  of  this  study  was  to  identify  factors  that  act  as  barriers  to 
implementing  KM  in  U.S.  military  organizations.  The  research  outlined  AFKN’s 
beginnings  in  the  early  1990s  as  an  on-line  acquisition  regulations  repository  for  the 
systematic  procedures  for  conducting  acquisitions,  as  well  as  other  miscellaneous  pieces 
of  information  such  as  acquisitions  points  of  contact  and  lessons  learned.  After  its  initial 
success,  the  Special  Programs  Office  (SPO)  proposed  use  of  the  system  across  the 
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Department  of  Defense.  The  Office  of  the  Under  Secretary  of  Defense  for  Acquisition 
Technology  approved  this  request  in  1998,  and  the  system  was  formally  named  the 
“Defense  Acquisition  Deskbook  program.”  This  program  was  managed  and  operated  by 
Air  Force  Material  Command  (AFMC),  specifically  the  DR  directorate  within  AFMC. 

By  mid-2000,  AFMC/DRA  had  evolved  into  four  distinctive  knowledge  management 
systems:  “the  AFKM  Lessons  Learned  database,  the  AFMC  portion  of  the  DoD 
Acquisition  Deskbook,  the  AFKM  Help  Center,  and  the  AFMC  Virtual  Schoolhouse.”  It 
is  important  to  note,  although  this  was  an  AFMC  effort,  designed,  and  used  primarily  by 
AFMC  personnel,  the  actual  title  of  the  program  was  “ Air  Force  Knowledge 


Management.”  See  Figure  1  for  a  sample  screen  shot  of  AFKM  at  that  point  in  time. 
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Figure  1  -  The  Air  Force  Knowledge  Management  Home  Page  (Circa  2001) 
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At  around  this  time,  the  first  iteration  of  CoPs  came  about.  At  first,  they  were 
called  “Workspaces.”  As  you  can  see  in  Figure  2  below,  access  to  these  workspaces  was 
from  the  links  on  the  right  side  of  the  AFKM  home  page.  Initially  these  workspaces  were 
custom  built  for  each  specific  group. 
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Figure  2  -  AFKM  Community  of  Practice  Workspaces 
Bartczak  found  numerous  barriers  towards  organizational  knowledge 
management,  to  include  a  lack  of  leadership  commitment  and  reinforcing  behaviors. 
Additionally,  she  noted  several  coordinating  and  control  barriers  that  had  hampered 
AFKM’s  development.  (Bartczak,  2002)  Her  findings  were  divided  into  the  four 
“managerial  influence  factors”  of  leadership,  coordination,  control,  and  measurement. 
The  findings  from  these  specific  areas  are  given  in  Table  1  below. 
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Influence  Factor 

Finding 

Leadership 

•  Lack  of  leadership  commitment  at  critical  levels 

•  Lack  of  reinforcing  behaviors 

Coordination 

•  AFKM  name  conflict 

•  Uncoordinated  evolution  of  AFMC  and  AF  KM  programs 

•  Conflict  with  IT  organization 

Control 

•  Lack  of  control  of  contractors 

•  Restrictive  impact  of  external  control  policies 

•  Re-aligning  technical  focus 

•  Shaping  the  AFMC  program  team 

•  Cultivating  partnerships 

•  Limiting  information  access 

•  Restrictive  software  procurement/use  policy 

•  Absence  of  information/knowledge  quality  controls 

Measurement 

•  Measurements  needed  to  gain/keep  leadership  support 

•  Lack  of  appropriate  measures 

Table  1  -  AFMC  Barriers  to  Knowledge  Management  (Bartczak,  2002) 


Over  the  years,  AFKN  has  continued  to  grow  to  its  current  state  of  over  1000 
active  CoPs.  See  Figure  3  for  a  screen  shot  of  the  current  AFKN  home  page. 
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What  is  a  Community  of  Practice?  ™  Example 
A  Community  of  Practice  (CoP)  workspace  provides  a  web-based  collaborative 
environment  where  members  of  a  group  use  shared  information  and  administrative  and 
communications  tools  to  conduct  business,  manage  a  project,  keep  abreast  of  important 
group  issues  and  solve  group  problems.  You  can  choose  your  CoP  capabilities,  revise 
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Figure  3  -  AFKN  Home  Page  (Circa  2004) 
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The  workspaces  have  also  matured  to  what  we  currently  call  CoPs.  These  CoPs 
are  fairly  generic  with  a  minimum  of  customizability.  This  is  in  contrast  to  the  first 
AFKM  workspaces  mentioned  previously  that  were  custom  built.  See  Figure  4  below 
for  a  sample  AFKN  CoP. 
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Figure  4  -  Sample  AFKN  CoP 


May  (2003)  followed  up  on  Bartczak’s  work,  and  performed  a  study  of  the 
evolution  of  AFKN  CoPs.  This  research  sought  to  identify  the  stages  of  maturity  of  the 
various  CoPs  based  on  McDermott’s  Theories.  Based  on  these  theories,  May  set  out  to 
identify  the  AF/AFMC  CoP’s  “perceived  stage”  of  development.  These  stages  are  1) 
Potential,  2)  Building,  3)  Engaged,  4)  Active,  and  5)  Adaptive.  In  this  research,  May 
surveyed  all  AF/AFMC  CoP  knowledge  owners  and  administrators.  Not  surprisingly, 
this  research  concluded,  “on  average,  the  AF/AFMC  CoPs  are  in  the  very  early  stages  of 
evolution.”  See  Figure  5  for  the  results  to  this  study.  (May,  2003) 
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Stage  1  Stage  2  Stage  3  Stage  4  Stage  5 
Potential  Building  Engaged  Active  Adaptive 


Figure  5  -  Stages  of  CoP  Development  (May,  2003) 

From  these  findings,  May  asserts  that  there  is  a  wide  range  of  actions  that  can  be 
taken  to  improve  the  efficiency  and  effectiveness  of  existing  CoPs.  Some  of  his 
recommendations  include  increasing  leadership  involvement  and  support,  as  well  as 
membership  education  and  training.  Additionally,  he  recommends  to  more  clearly  define 
the  purpose  and/or  objectives  of  each  CoP.  Finally,  he  suggests  improving  the 
technology  tools  for  navigating  the  CoP  collaborative  workspace.  (May,  2003) 

Rodriquez’s  (2004)  thesis  researched  looked  at  content  management  issues  within 
AFKN  CoPs.  Content  management  involves  identifying,  collecting,  and  managing 
content  within  an  organization.  It  should  provide  a  standardized  approach  for  content 
ownership,  use,  storage,  and  classification.  As  defined  in  this  research,  content 
management  is: 

“...a  practice  to  provide  meaningful  and  timely  information  to  end  users  by 
creating  processes  that  identify,  collect,  categorize,  and  refresh  content  using  a  common 
taxonomy  across  the  organization”  (Rodriguez,  2004) 
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This  study  was  accomplished  by  performing  multiple  case  studies  on  eight  active 
AFKN  CoPs.  Rodriquez  found  that  having  a  “well-developed”  taxonomy  is  essential  for 
good  content  management.  He  also  pointed  out  that  the  knowledge  owner  was  critical  to 
the  validation  of  the  relevance  and  currency  of  the  data  on  their  CoP.  Some  other 
recommendations  that  he  had  included  utilizing  taxonomy  experts,  developing  content 
management  guidelines,  conducting  reoccurring  content  audits,  focusing  on  the  users,  not 
focusing  on  the  technological  solution.  (Rodriguez,  2004)  See  Table  2  below  for  a 
summary  of  the  issues,  actions  taken  and  suggestions  for  improvement  from  this  study. 


L  What  are  the  content  management  issues  associated  with  the  AF  CoPs  hosted  by _ 

•  There  is  a  lack  of  documented  content  management  processes  and  procedures  by  the  CoPs. 

•  The  CoPs  have  had  no  driving  need  to  purge  outdated  content  since  there  is  no  pressure  on  limiting  the 
amount  of  content  stored  on  the  CoPs. 

•  CoP  administrators  placed  little  emphasis  on  purging  or  formally  archiving  outdated  content  since  no 
limitation  exists  on  the  amount  of  content  stored  by  a  CoP  and  these  content  management  processes  are 
not  an  immediate  priority. 

•  Not  all  CoP  administrators  are  volunteers  or  have  CoP  administration  as  their  primary  duty _ 

2.  What  are  the  CoP  content  management  issues  critical  to  success  as  identified  by  AF  CoPs 

knowledge  owners/members? _ 

•  Need  a  consistent  taxonomy  for  the  CoP. 

•  The  responsibility  for  the  file  structure  of  the  CoPs  is  left  to  the  CoP  administrators. 

•  Each  CoP  either  identified  getting  knowledge  owners  trained  as  a  critical  issue  or  mentioned  it  as  an 
action  taken  to  meet  their  content  management  issues. 

•  Training  will  be  a  reoccurring  issue  as  people  move  jobs  and  new  knowledge  owners  come  on  board 

3.  What  actions  have  AFMC/DRW  or  the  AF  CoPs  themselves  taken  to  address  content  management 

issues? _ 

•  The  CoP  members  are  taking  it  upon  themselves  to  build  a  taxonomy  based  on  the  experience  of  the 
knowledge  owner  with  the  content  on  the  site. 

•  The  Knowledge  Now  team  provides  basic  guidelines  for  the  establishment  of  a  CoP’s  initial  taxonomy 

•  Additional  help  for  creating  an  initial  taxonomy  has  been  made  available  in  training  workshops. 

•  In  addition,  the  Knowledge  Now  team  has  provided  a  tool  to  give  alerts  (based  on  documents  a  user 

selects)  on  changing  documents. _ 

4.  What  suggestions  or  solutions  do  AF  CoP  knowledge  owners/members  propose  to  solve  the  content 

management  problems  that  they  are  experiencing? _ 

•  Need  to  document  content  management  processes  and  procedures  based  on  industry  best  practices. 

•  Three  of  the  four  functional  CoPs  identified  not  having  the  time  or  resources  to  execute  good  content 
management  efforts  as  an  issue. 

•  Suggested  solutions  included  assigning  an  individual  to  add  new  content  in  a  consistent  manner  while 
removing  outdated  content. 

•  Other  alternatives  involved  hiring  a  support  contractor  to  execute  the  actions  required  for  good  content 

management  or  having  junior  members  maintain  the  site  with  the  help  of  more  senior  members. _ 

Table  2  -  Rodriguez  (2004)  Findings 
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Hinrichsen  (2004)  examined  12  cultural  factors  affecting  use  of  Communities  of 
Practice  (See  Figure  6Through  his  research,  Hinrichsen  explored  the  idea  that  culture 
internal  to  CoPs  influences  use.  His  hypothesis  was  that  people  who  are  member  of 
“higher  use”  CoPs  will  place  greater  emphasis  on  KM  culture  variables  than  those  who 
belong  to  “lower  use”  CoPs.  (Hinrichsen,  2004) 


Figure  6  -  Hinrichsen's  Initial  Research  Model 


Hinrichsen  surveyed  the  entire  population  of  AFKN  to  ascertain  his  findings.  Six 
thousand  one  hundred  and  twenty  five  individuals  were  contacted  for  the  survey,  with  a 
response  of  1,042  people,  for  a  response  rate  of  17%.  This  survey  was  a  “cross- 
sectional”  survey,  performed  with  Fitzgerald  (2004).  Hinrichsen’s  research  showed  that 
out  of  the  12  factors  examined,  there  were  only  “significant”  differences  in  information 
sharing  and  positive  culture.  Although  this  study  concluded  that  Shaw  and  Tuggle’s 
model  for  KM  culture  variables  in  CoPs  was  not  predictive  of  CoP  use,  he  felt  that 
factors  such  as  types  of  communities  or  stage  of  development,  might  show  a  stronger 
relationship  between  the  variables.  (Hinrichsen,  2004) 
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Fitzgerald’s  (2003)  thesis  work  highlighted  the  factors  affecting  knowledge 
transfer,  information  sharing,  and  technology  acceptance  in  AFKN  CoPs.  Fitzgerald 
identified  ten  specific  factors  (See  Figure  7).  This  research  was  based  on  Venkatesh  et  al. 
(2003). 


As  stated  previously,  the  survey  used  to  conduct  this  study  was  a  “cross- 
sectional”  survey  performed  with  Hinrichsen  (2004).  Of  the  ten  factors  originally  looked 
at,  the  research  concluded  that  the  factors  of  job  performance,  trust,  willingness  to  share, 
security  constraints,  and  facilitator  seemed  to  affect  participation  in  CoPs.  See  Figure  8 
for  the  revised  model. 
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I  Job  Performance 
|  Willingness  to  Share 
I^SecurityConstramts 
Trust 
F  acilitator 

Filntul  Ut4«it>n  CaT  Size,  UiMxr y ’/Pfw  X(i  k  ta  i  y  EnployintiU  Category  J 

- ►  Indicates  significant  deference  at  p  =  0.05  between  high  and  low  quartiles  of  CoP  usage 

. ►  Indicates  significant  difference  at  p  =  0.05  between  Group  1  and  Group  6  of  CoP  usage 

Figure  8  -  Fitzgerald's  Revised  Research  Model 

These  findings  answered  Fitzgerald’s  first  research  question;  “Can  we  identify 
factors  that  affect  participation  between  high  and  low  use  AFKN  Communities  of 
Practice?”  The  second  research  question:  “What  differentiates  the  successful  and 
unsuccessful  AFKN  hosted  Communities  of  Practice?”  was  not  fully  answered.  Initially, 
this  study  based  success  on  the  amount  of  participation  in  the  CoP.  Fitzgerald  concluded 
that  the  more  “successful”  CoPs  display  a  greater  positive  perception  of  the  five  factors 
found  to  affect  participation.  (Fitzgerald,  2004) 

As  stated  in  the  opening  section  of  this  chapter,  there  is  a  disparity  between  the 
definition  of  a  CoP  between  the  current  research  and  the  previous  AFKN  CoP  research. 
Additionally  there  have  been  numerous  findings  identified  in  the  previous  AFKN  CoP 
research  (see  Table  3  for  a  summary  of  findings).  Based  on  the  current  research’s 
definition  of  CoPs  and  the  findings  and  recommendations  of  these  previous  AFKN  CoP 
studies,  further  study  of  AFKN  CoPs  as  a  form  of  information  technology  is  needed 
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Researcher 

Research  Area 

Research 

Method 

Findings/Recommendations 

Bartczak 

(2002) 

-  Identification  of  influence 
factors  that  act  as  barriers 
to  implementing  KM  in 

U.S.  military  organizations 

Case  Study 

Found  numerous  barriers  towards  organizational  knowledge  management,  to  include  a 
lack  of  leadership  commitment  and  reinforcing  behaviors. 

May  (2003) 

-  Evaluating  CoPs  to 
identify  their  “Current 
state  of  evolution” 

Survey  (All 
AFKN  CoP 
Users) 

Research  concluded,  “on  average,  the  AF/AFMC  CoPs  are  in  the  very  early  stages  of 
evolution.” 

Recommendations  include: 

-  Increasing  leadership  involvement,  support,  membership  education  and  training 

-  More  clearly  define  the  purpose/objectives  of  each  CoP 

-  Improving  the  technology  tools  for  navigating  the  CoP  collaborative  workspace 

Rodriquez 

(2004) 

-  Content  management 
issues  with  CoPs 

Case  Study 
(8  CoPs 

-  Having  a  “well-developed”  taxonomy  is  essential  for  good  content  management. 

-  Noted  that  the  knowledge  owner  was  critical  to  the  validation  of  the  relevance  and 

currency  of  the  data  on  their  CoP 

-  Some  other  recommendations  that  he  had  included  were: 

—  Utilizing  taxonomy  experts, 

—  Developing  content  management  guidelines, 

—  Conducting  reoccurring  content  audits, 

—  Focusing  on  the  users,  not  on  the  technological  solution 

Fitzgerald 

(2004) 

-  Research  highlighted  the 
factors  affecting 
knowledge  transfer, 
information  sharing,  and 
technology  acceptance  in 
AFKN  CoPs. 

Survey  (All 
AFKN  CoP 
Users) 

-Of  the  ten  factors  that  he  originally  looked  at  (See  Figure  7)  research  concluded  that  only 
job  performance,  willingness  to  share,  security  constraints,  Trust,  and  facilitator  affect 
participation  in  CoPs 

-  Initially,  this  study  based  success  on  the  amount  of  participation  in  the  CoP. 

-  Fitzgerald  concluded  that  the  more  “successful”  CoPs  display  a  greater  positive 

perception  of  the  five  factors  found  to  affect  participation. 

Hinrichsen 

(2004) 

-  Examined  the  cultural 
factors  affecting  use  of 

CoPs 

Survey  (All 
AFKN  CoP 
Users) 

-  Explored  the  idea  that  culture  internal  to  CoPs  influences  use 

-  Out  of  the  12  factors  examined  (See  Figure  6),  only  differences  in  information  sharing 

and  positive  culture. 

-  For  additional  research,  felt  that  using  other  factors  might  show  a  stronger  relationship 

between  the  variables  such  as  types  of  communities  or  stage  of  development 

Table  3  -  Summary  of  AFKN  Research 


Technology  Acceptance 

As  stated  in  chapter  one,  this  study  will  look  at  AFKN  CoPs  based  on  the 
technology  acceptance  model  (TAM).  Based  on  the  previous  sections  overview  of  the 
previous  studies  of  the  AFKN  CoPs,  with  the  exception  of  Fitzgerald  (2004),  all  of  the 
other  studies  looked  at  the  CoPs  from  either  an  organizational  behavior  perspective  and 
not  from  an  information  technology  perspective.  Even  Fitzgerald’s  study  did  not  tackle 
TAM  directly,  but  instead  it  studied  knowledge  transfer,  information  sharing,  and 
technology  acceptance  together.  At  this  juncture,  it  is  important  to  recognize  that  one  of 
the  greatest  concerns  for  information  systems  research  and  practice  is  the  adoption  and 
use  of  information  technology.  Therefore,  it  is  essential  that  technology  acceptance  of 
AFKN  CoPs  be  the  primary  focus  of  this  research.  Venkatesh  and  Davis  (2000)  assert 
that  understanding  and  creating  the  conditions  that  influence  human  organizations  to 
embrace  information  systems  remains  a  high-priority  research  issue.  (Venkatesh  & 

Davis,  2000) 

The  technology  acceptance  model  seeks  to  provide  an  explanation  of  the 
determinants  of  computer  acceptance  that  is  “general,  capable  of  explaining  user  behavior 
across  a  broad  range  of  end-user  computing  technologies  and  user  populations,  while  at 
the  same  time  being  both  parsimonious  and  theoretically  justified.”  (Refer  to  Figure  9) 
The  TAM  is  based  on  the  belief  that  perceived  usefulness  and  perceived  ease  of  use  are 
“of  primary  relevance  for  computer  acceptance  behaviors.”  (Davis,  Bagozzi,  &  Warshaw, 
1989) 
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Figure  9  -  The  Technology  Acceptance  Model 

The  explosion  in  Internet  usage  and  huge  government  funding  initiatives  in  digital 
libraries  has  drawn  attention  to  research  on  digital  libraries.  Whereas  the  traditional 
focus  of  digital  library  research  has  been  on  the  technological  development,  there  is  now 
a  call  for  user-focused  research.  Although  millions  of  dollars  have  been  spent  on  these 
systems,  potential  users  may  not  use  the  systems  in  spite  of  their  availability.  Using  the 
technology  acceptance  model  as  a  theoretical  framework,  Hong  et  al.  (2001/2002)  studies 
the  effect  of  a  set  of  individual  differences  and  system  characteristics  on  intention  to  use 
digital  libraries.  Digital  libraries  provide  easier  tracking  of  digital  media;  remote,  fast 
and  fair  access  to  its  collections;  and  increased  flexibility  and  power  to  users.  This  study 
identified  a  strong  relationship  between  relevance,  that  is  to  say,  the  data  within  the 
library  was  relevant  to  what  the  users  needed,  and  perceived  usefulness.  (Hong,  Thong, 
Wong,  &  Tam,  2001/2002) 

Since  this  initial  use  of  the  TAM,  it  has  seen  many  iterations.  In  2002,  Venkatesh  et  al. 
developed  a  model  to  examine  the  influence  of  pre-training  and  training  environment 
interventions  (termed  users  acceptance  enablers).  See  Figure  10  below;  The  Integrated 
Model  of  Continued  Technology  Usage.  The  goal  of  this  research  was  to  understand  how 
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user  perceptions  are  formed  prior  to  system  implementation. 


Figure  10  -  Integrated  Model  of  Continued  Technology  Usage 

This  study  concluded  that  interventions  in  both  pre -training  and  training 
environments  played  a  “pivotal”  role  in  shaping  the  users  initial  motivations  and 
perceptions.  This  in  turn  formed  what  the  researchers  felt  were  the  basis  for  intentions 
and  use  over  time.  Furthermore,  they  noted  a  strong  direct  and  indirect  influence  of  ease 
of  use  and  intrinsic  motivation,  and  concluded  that  technology  acceptance  initiatives 
should  focus  on  interventions  designed  to  increase  perceptions  that  the  technology  is  easy 
and  enjoyable  to  use.  (Venkatesh,  Speier,  &  Morris,  2002) 

In  2003,  Venkatesh  et  al.  reviewed  eight  prominent  models  within  the  study  of 
understanding  individual  acceptance  of  new  IT.  Their  goal  was  to  identify  similarities  as 
well  as  differences  between  the  models.  See  Figure  1 1  below;  the  Unified  Theory  of 
Acceptance  and  Use  of  Technology  (UTAUT).  This  model  seeks  to  tie  all  of  the  major 
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Figure  11  -  Unified  Theory  of  Acceptance  and  Use  of  Technology  (UTAUT) 

technology  acceptance  issues  together  into  a  cohesive  model.  In  testing  the  model, 
Venkatesh  et  al.  found  these  tests  provided  strong  empirical  support  for  UTAUT,  which 
posits  three  direct  detenninants  of  intention  to  use  (performance  expectancy,  effort 
expectancy,  and  social  influence)  and  two  direct  determinants  of  usage  behavior 
(intention  and  facilitating  conditions).  One  of  the  recommendations  from  this  research  is 
the  adoption  of  the  UTAUT  model  to  other  technologies  such  as  collaborative  systems. 
(Venkatesh  et  al.,  2003) 

Although  there  is  an  extensive  body  of  literature  that  addresses  TAM,  the 
application  of  TAM  to  collaborative  or  knowledge  management  systems  is  limited. 
Dasgupta  et  al.  (2002)  studied  the  use  of  TAM  with  e-collaboration  technology.  This 
study  took  the  basic  framework  from  Davis  (1989)  and  applied  it  in  order  to  validate  the 
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findings  against  the  collaborative  system  (see  Figure  12  for  the  research  model)  that  was 
being  used  in  a  classroom  setting  for  instructor  and  class  for  course  communications. 


Figure  12  -  Dasgupta  et  al.  (2002)  Research  Model 


In  this  study,  perceived  ease  of  use  and  usefulness  were  measured  using  a  12- 
question  survey  based  on  Davis’  1989  instrument.  Level  is  a  dummy  variable  used  to 
identify  novice  and  advanced  users.  Use  was  obtained  from  system  logs  that  tracked 
usage.  Finally,  the  perfonn  variable  was  obtained  from  a  “weighted  average  of  scores 
from  assignments,  exams  and  projects.”  (Dasgupta,  Granger,  &  McGarry,  2002) 

The  findings  from  this  study  generally  support  the  previously  mentioned  studies 
of  TAM.  Some  areas  that  were  supported  include  the  positive  influence  that  perceived 
ease  of  use  has  on  perceived  usefulness;  past  experience  has  a  positive  influence  on 
system  usage;  and  use  of  the  system  has  a  positive  influence  on  user  perfonnance. 
Additionally,  it  was  noted  that  perceived  ease  of  use  does  not  have  a  “significant  effect” 
on  Usage.  An  important  finding  from  this  research  that  contradicts  previous  findings  in 
regards  to  TAM  was  the  negative  relationship  that  perceived  usefulness  has  on  use  of  the 
system.  The  researchers  in  this  study  believe  that  this  finding  is  directly  related  to 
experience  with  the  system;  whereas  the  more  familiar  a  user  was  with  the  system,  the 
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faster  and  more  efficiently  they  could  perfonn  a  particular  task  with  a  minimum  amount 
of  page  hits.  This  theory  was  supported  by  the  significant  differences  in  usage  between 
novices  and  advanced  users.  (Dasgupta  et  ah,  2002) 

The  Research  Model 

Of  the  five  previous  studies  performed  on  AFKN  CoPs,  only  Fitzgerald  (2004) 
looked  at  how  usage  of  CoPs  based  on  the  technology  acceptance  model.  The  model  for 
the  current  research  is  drawn  from  the  above-mentioned  TAM  research.  Please  refer  to 
Figure  13  for  the  below  discussion  of  the  research  model. 


Figure  13  -  Current  Research’s  Initial  Model 

The  base  for  the  research  model  is  the  1989  TAM  model.  (Davis  et  ah,  1989)  The 
four  key  items  that  came  out  of  this  model  are  Perceived  Usefulness  (U),  Perceived  Ease 
of  Use  (EOU),  Intention  to  Use  (IU),  and  Usage  Behavior  (UB).  Davis  concluded  that 
perceived  usefulness  is  a  major  determinant  of  people’s  intention  to  use.  (Davis  et  ah, 
1989)  In  another  study,  Venkatesh  et  al.  (2002)  stated  that  ease  of  use  has  a  “strong 
influence”  on  intention  to  use.  (Venkatesh  et  al.,  2002)  He  also  noted  that  the  influence 
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that  ease  of  use  has  on  use  provides  a  significant  secondary  affect  on  intention  to  use  and 
that  intention  to  use  is  a  “major  determinant  of  usage  behavior.”  Davis  concluded  that 
usage  behavior  “can  be  predicted  reasonably  well  from  their  intentions.”  (Davis  et  ah, 
1989) 

From  this  base,  User  Acceptance  Enablers  (UAE)  was  added  because  of  its 
“pivotal  role”  in  the  user’s  initial  “motivations  and  perceptions”  that  in  turn  forms  the 
basis  for  “intentions  and  technology  use  over  time.”  (Venkatesh  et  ah,  2002)  Based  on 
previous  AFKN  studies  (Bartczak,  2002;  Fitzgerald,  2004;  Hinrichsen,  2004),  Social 
Influences  (SI)  was  added  to  the  model.  The  UTAUT  showed  “strong  empirical  support” 
for  social  influence  as  a  direct  detenninant  of  intention  to  use.  They  noted  that  social 
influence  is  “more  likely  to  be  important”  in  systems  that  are  mandatory  to  use.  Finally, 
the  UTAUT  showed  the  direct  detenninants  of  usage  behavior  to  be  intention  to  use  and 
facilitating  conditions  (FC).  (Venkatesh  et  ah,  2003) 

Chapter  Summary 

This  chapter  defined  what  CoPs  are  and  described  some  of  the  factors  that  affect 
knowledge  transfer  and  acceptance  of  this  technology.  The  chapter  also  defined  CoPs 
and  their  uses  with  KM,  reviewed  the  previous  studies  of  AFKN  KM  efforts,  and  also 
examined  the  literature  in  regards  to  KM  and  technology  acceptance.  The  chapter 
concluded  with  a  detailed  description  of  the  current  research  model. 
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III.  Methodology 


Overview 

This  chapter  describes  the  methodology  used  in  conducting  this  research  project. 
It  will  be  broken  into  three  main  sections.  First,  the  research  method  selection  will  be 
covered  in  detail.  This  step  is  important  to  show  how  the  researcher  went  from  a  very 
high-level  look  at  the  problem,  down  to  a  focused  perspective  that  provided  the 
framework  from  which  the  research  will  be  performed.  Second,  the  quantitative  portion 
of  this  research  will  be  addressed.  This  area  will  describe  how  the  survey  data  was 
collected  during  a  previous  research  effort  as  well  as  show  how  the  data  will  be 
examined.  Third,  the  design  of  the  qualitative  portion  of  this  study  will  be  covered  to 
show  the  procedures  used  to  perform  the  case  study  aspect  of  this  research. 

Research  Method  Selection 

The  selection  for  the  methodology  of  this  research  is  primarily  based  on  the 
recommendations  of  Creswell.  (Creswell,  2003)  He  suggests  addressing  three  questions 
that  he  feels  are  central  to  the  “Elements  of  Inquiry.”  These  questions  are: 

1 . )  What  knowledge  claims  are  being  made  by  the  research? 

2. )  What  strategies  of  inquiry  will  inform  the  procedures? 

3. )  What  methods  of  data  collection  and  analysis  will  be  used? 

Based  on  the  answers  to  these  questions,  a  researcher  can  decide  upon  the  appropriate 
“Approach  to  Research.”  The  selected  approach  acts  as  a  framework  to  guide  the 
researcher  in  collecting,  analyzing,  documenting,  and  validating. 

1.)  What  knowledge  claims  are  being  made  by  the  research? 

Creswell  (2003)  recommends  identifying  a  specific  knowledge  claim  based  on 
assumptions  about  what  the  researcher  expects  to  learn  during  their  inquiry.  Based  on 
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this  guidance,  the  specific  research  method  for  this  research  will  be  from  a  “Pragmatism” 
style  due  to  this  claims  real-world  practice  orientation  and  the  need  to  identify  what 
works.  Furthermore,  he  feels  that  pragmatist  researchers  look  to  the  “what”  and  “how”  of 
research,  based  on  its  intended  consequences.  (Creswell,  2003) 

2. )  What  strategies  of  inquiry  will  inform  the  procedures? 

Based  on  the  pragmatic  knowledge  claim  from  question  one,  the  answer  to 
question  two  is  to  use  the  mixed  methods  strategy  of  inquiry.  Recognizing  that  all 
methods  have  limitations,  Creswell  felt  that  the  biases  that  are  inherent  in  any  single 
method  could  neutralize  or  cancel  the  biases  of  other  methods.  This  research  will  take 
the  “Concurrent  Procedures”  strategy  of  bringing  the  data  together.  In  this  method,  the 
researcher  “converges”  quantitative  and  qualitative  data  in  order  to  provide  a  more 
comprehensive  analysis  of  the  research  problem.  In  this  design,  the  investigator  collects 
both  fonns  (quantitative  and  qualitative)  of  data  at  the  same  time,  and  then  integrates  the 
information  in  the  interpretation  of  the  overall  results.  (Creswell,  2003) 

3. )  What  methods  of  data  collection  and  analysis  will  be  used? 

Based  on  the  selection  of  mixed  method  strategy,  there  are  several  choices  to  use  for 
collecting  data.  This  research  will  incorporate  a  pre-interview  survey  based  on 
Venkatesh  et  al.  (2003),  a  structured  interview  that  includes  both  open  and  closed  ended 
questions  as  well  as  data  from  a  previous  research  study  on  AFKN  CoPs  (see  Figure  14). 
This  method  will  allow  the  researcher  to  converge  on  the  broad  results  in  order  to  focus 
on  detailed  views  from  the  participants.  (Creswell,  2003) 
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Figure  14  -  Mixed  methods  strategy 
Quantitative  Research  Design 

The  methodology  applied  in  this  research  will  mirror  the  methodology  used 
during  the  Fitzgerald  and  Ffinrichsen  (2004)  studies.  In  these  studies,  the  research 
designs  centered  around  a  survey.  The  survey  was  cross-sectional,  in  order  to  gather  data 
for  both  research  projects  without  having  to  survey  the  same  population  twice  for  similar 
information.  Emails  requesting  participation  in  the  survey  were  sent  to  all  6165 
registered  members  of  the  120  CoPs  fitting  the  sample  criterion.  One  thousand  forty  two 
people  took  the  survey,  for  a  response  rate  of  17%.  Similar  to  the  current  study,  the 
survey  was  designed  to  collect  both  quantitative  and  qualitative  data.  The  constructs 
were  measured  using  a  seven-point  Likert-scale,  indicating  one  as  “Strongly  Disagree” 
and  seven  as  “Strongly  Agree.”  See  Figure  15  below,  for  a  sample  question  from  the 
survey  with  the  accompanying  Likert  Scale. 


2.  Information  is  shared  in  my  CoP. 

Strongly 

Disagree 

Disagree 

Slightly 

Disagree 

Don't  know 

Slightly  Agree 

Agree 

Strongly  Agree 

Figure  15  -  Sample  Survey  Question  (Flinrichsen,  Fitzgerald  2004) 
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To  assess  the  qualitative  nature  of  the  research,  two  open-ended  questions  were 
asked  at  the  end  of  the  survey.  All  quantitative  questions  were  developed  to  infer 
increased  participation  as  Likert  scale  responses  increased,  with  the  exception  of  the 
anonymity  and  security  constraint  question.  If  a  respondent  answered  a  question  with  a 
seven,  the  corresponding  inference  should  indicate  positive  affect  on  participation.  Three 
demographic  questions  were  asked  to  determine  the  community  each  respondent  was  a 
member  of,  determine  respondent’s  rank  or  grade,  and  respondent’s  length  of  time  as  a 
CoP  member. 

The  communities  of  practice  (CoPs)  were  rank  ordered  by  usage  (page  hits  per 
member  over  a  last  three  month  period)  and  then  divided  into  six  equal  groups.  Each 
group  contained  twenty  CoPs  and  the  groups  were  numbered  from  1  to  6,  with  Group  1 
containing  the  CoPs  with  the  highest  usage  rates,  and  so  on,  down  to  Group  6,  which 
contained  the  twenty  CoPs  with  the  lowest  usage  rates.  This  also  provided  a  quick  visual 
method  to  assess  how  the  group’s  response  rates  compared.  The  CoPs  were  divided  into 
the  six  equal  groups  for  the  emails  sent  to  each  CoP  group  member  to  contain  a  survey 
link  with  a  smaller  amount  of  CoPs  to  scroll  down  to  in  order  to  find  the  CoP  belonging 
to  each  participant  (demographic  question  #1).  The  response  numbers  for  each  group  are 
shown  in  Figure  16. 
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Figure  16  -  CoP  Member  Response  Rates  by  Group 


The  length  of  time  participants  belonged  to  their  respective  CoP  is  provided  in 
Table  4.  An  overwhelming  number  of  participants  (66.7  %)  have  belonged  to  their  CoPs 
for  12  months  or  less,  which  corresponds  to  the  increased  interest  throughout  the  military 
in  using  CoPs  as  a  knowledge  management  tool. 


Months  as  CoP 

Member 

Frequency  Percent 

1  12 

610 

66.7 

13  24 

129 

14.1 

25  36 

25 

2.7 

Less  than  1 

86 

9.4 

More  than  36 

65 

7.1 

Total 

915 

100.0 

Table  4  -  Length  of  CoP  Membership 


Lastly,  the  breakdown  of  rank  and  grade  for  participants  is  provided  in  Table  5. 
Almost  half,  45.6%,  the  survey  participants  fell  within  the  grades  of  GS-1 1  through  15. 
Next  were  senior  non-commissioned  officers  at  1 1 .7%,  contractors  at  1 1 .6%,  and  field 
grade  officers  at  10.8%.  While  these  numbers  bode  well  for  the  level  of  experience 
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present  in  AFKN  CoPs,  the  lack  of  participation  by  lower  ranks  and  grades  could  indicate 
a  missed  opportunity  for  the  CoP  members  in  need  of  the  knowledge  and  insight 
possessed  by  the  senior  CoP  members.  Another  explanation  for  this  might  be  the 
disproportionate  number  of  individuals  within  these  particular  grades  that  are  assigned  to 
Air  Force  Material  Command  and  is  therefore  an  accurate  representation  of  these  grades. 


Rank  or  Grade 

Frequency 

Percent 

Cumulative  Percent 

Contractor 

106 

11.6 

11.6 

El  E4 

4 

0.4 

12 

E5  E6 

62 

6.8 

18.8 

E7  E9 

107 

11.7 

30.5 

GS1  GS5 

40. 

4 

30.9 

GS1 1  GS15 

417 

45.6 

76.5 

GS6  GS10 

47 

5.1 

81.6 

01  03 

54 

5.9 

87.5 

04  06 

99 

10.8 

98.4 

07  010 

4 

0.4 

98.8 

Other 

11 

1.2 

100 

Total 

915 

100 

Table  5  -  Response  Frequency  by  Rank  and  Grade 


The  major  limitation  of  this  research  was  its  lack  of  blocking  for  the  factors  of: 
functional  organization,  formality,  access,  grade/position,  and  time.  Therefore,  to  answer 
the  following  research  questions,  the  original  data  will  be  re-examined  using  blocking 
factors  for  these  five  factors: 

la.  Based  on  existing  models,  is  there  a  difference  between  factors  based  on 
whether  the  CoP  is  used  by  teams,  function,  or  directorates? 

lb.  Based  on  existing  models,  is  there  a  difference  between  factors  based  on 
whether  the  CoP  is  formed  informally  or  formally? 

lc.  Based  on  existing  models,  is  there  a  difference  between  factors  based  on 
whether  the  CoP  is  open  or  closed? 

ld.  Based  on  existing  models,  is  there  a  difference  between  factors  based  on  how 
long  the  individual  has  been  with  the  CoP? 
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le.  Based  on  existing  models,  is  there  a  difference  between  factors  based  on  the 
individual’s  grade/position? 

To  address  these  five  factors,  dummy  variables  will  be  created  and  added  to  the 
survey’s  results.  These  factors  will  be  used  to  account  for  the  variances  based  on  each 
specific  factor’s  influence  on  the  data  by  performing  analysis  of  variance  (ANOVA)  tests 
on  the  data. 

Qualitative  Research  Design 

After  choosing  the  case  study  as  the  primary  research  strategy,  the  design  of  the 
research  effort  was  developed.  The  design  of  the  research  effort  is  very  critical  in 
determining  “what  questions  should  be  addressed,  the  type  of  data  to  collect,  and  how  to 
analyze  the  data.”  (Yin,  2003)  According  to  Yin,  there  are  five  components  of  a 
research  design:  1)  research  questions,  2)  propositions,  3)  unit  of  analysis,  4)  logic 
linking  the  data  to  the  propositions,  and  5)  criteria  for  interpreting  the  findings.  (Yin, 
2003)  The  next  section  addressed  the  five  components  of  the  current  research  design 
method  that  will  be  used  to  answer  the  following  qualitative  research  questions: 

2a.  What  are  the  specific  factors  that  encouraged  an  individual  to  participate  in  a 
particular  CoP  when  initially  starting  to  use  the  CoP? 

2b.  What  factors  discouraged  an  individual  from  participating  in  a  particular  CoP 
when  initially  starting  to  use  the  CoP? 

3a.  What  factors  encouraged  an  individual  to  participate  in  a  particular  CoP  after 
initial  use? 

3b.  What  factors  discouraged  an  individual  from  participating  in  a  particular  CoP 
after  initial  use? 

Components  of  Research  Design 

(1)  Research  Questions.  As  stated  earlier  in  this  chapter,  a  mixed  method  case 
study  research  addresses  the  research  questions  in  this  study. 
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(2)  Research  Propositions.  The  research  propositions  present  the  purpose  of  the 
research.  The  proposition  has  to  address  the  purpose  of  the  study.  (Yin,  2003)  The 
purpose  of  this  research  is  to  identify  key  factors  affecting  acceptance  and  use  of  AFKN 
CoPs  and  then  to  discover  how  these  CoPs  are  being  used. 

(3)  Unit  of  Analysis.  The  unit  of  analysis  defines  what  the  “case”  is.  (Yin,  2003) 
The  unit  of  analysis  in  this  study  is  the  individual  CoP  users.  Although  this  study  is 
setting  out  to  identify  specifics  about  CoPs,  it  is  the  users  who  are  ultimately  “using”  and 
“accepting”  the  CoPs. 

(4)  Logic  Linking  the  Data  to  the  Propositions.  Multiple  sources  of  evidence  are 
a  result  of  the  developments  of  converging  lines  of  inquiry.  (Yin,  2003)  Data  for  case 
studies  can  come  from  many  sources  to  include  documentation,  archival  records, 
interviews,  direct  observations,  participant-observation,  and  physical  artifacts.  (Yin, 
2003)  This  study  uses  previous  survey  data,  current  metrics  of  CoPs,  interviews,  and 
direct  observations. 

(5)  Criteria  for  Interpreting  the  Research’s  Findings.  Data  collected  in  case  study 
research  is  hard  to  analyze  due  to  imprecise  strategies  and  techniques.  (Yin,  2003)  Yin 
(2003)  details  three  strategies  for  analyzing  data:  1)  relying  on  theoretical  propositions,  2) 
setting  up  a  framework  based  on  rival  explanations,  and  3)  developing  case  descriptions. 
The  strategy  chosen  in  this  research  relies  on  existing  theoretical  propositions.  Research 
on  acceptance  and  use  (primarily  TAM)  provided  the  basis  for  data  collection  and  served 
as  a  guide  to  analyze  the  collected  data. 
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Quality  of  the  Research  Design 

The  quality  of  the  chosen  research  design  detennines  the  reliability  and  validity  of 
the  study.  According  to  Yin  (2003),  four  tests  establish  the  quality  of  any  empirical 
social  research:  1)  construct  validity,  2)  internal  validity,  3)  external  validity,  and  4) 
reliability.  Case  studies  are  a  form  of  empirical  social  research  so  the  four  tests  are  also 
relevant  (Yin,  2003).  Table  6  lists  the  four  tests  used  in  this  research  to  address  the  test, 
and  the  phase  in  which  the  behaviors  are  used  during  the  research.  The  current  research 
uses  the  tactics  identified  in  Table  6  to  ensure  the  research  design  is  reliable  and  valid. 


Tests 

Case  Study  Tactic 

Phase  of  Research  in 
Which  Tactic  Occurs 

Construct  validity 

Use  multiple  sources  of 
evidence 

Data  collection 

Internal  validity 

Pattern  matching 

Data  Analysis 

External  validity 

Theory  in  a  single-case 
study 

Research  design 

Reliability 

Use  case  study  protocol 

Data  collection 

Table  6  -  Case  Study  Tactics  for  Design  Tests 

Construct  Validity 

Establishing  the  correct  operational  measures  for  the  concepts  being  studied 
addresses  construct  validity.  (Yin,  2003)  Yin  (2003)  presents  several  tactics  to  increase 
construct  validity  in  case  studies:  1)  use  multiple  sources  of  evidence  to  encourage 
convergent  lines  of  inquiry  which  is  used  during  data  collection;  and  2)  establish  a  chain 
of  evidence  that  is  applicable  during  data  collection.  The  current  research  uses  multiple 
sources  of  evidence  from  interview  transcripts  and  documentation  from  previous  research 
on  AFKN  CoPs. 
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Internal  Validity 

Establishing  a  causal  relationship  addresses  internal  validity.  (Yin,  2003)  Yin 
(2003)  identifies  pattern  matching  as  a  tactic  to  ensure  the  internal  validity  of  the 
research.  This  research  will  utilize  this  tactic  presented  by  Yin  (2003).  Pattern  matching 
will  be  used  to  analyze  interview  transcripts  and  documentation  gathered  from  previous 
research  to  locate  trends  in  the  data. 

External  Validity 

External  validity  is  addressed  through  the  generalizability  of  the  research.  (Yin, 
2003)  The  focus  of  this  research  is  acceptance  and  use  of  CoPs.  As  stated  in  Chapter 
two,  there  have  been  numerous  studies  of  CoPs  as  well  as  technology  acceptance.  The 
data  collected  from  this  research  will  be  compared  against  similar  research  in  this 
existing  literature.  Trends  developed  from  the  data  analysis  of  interviews  and  gathered 
documentation  will  be  crosschecked  with  literature  on  technology  acceptance  as  well  as 
other  AFKN  CoP  research. 

Reliability 

The  goal  of  reliability  is  to  minimize  the  errors  and  biases  in  a  study.  (Yin,  2003) 
Reliability  in  the  context  of  case  studies  is  ensuring  that  the  study  is  repeatable.  Human 
Subjects  Review  Protocol  documentation,  prepared  by  the  researcher,  details  the  specific 
execution  of  this  study.  Appendix  A  contains  the  approved  Human  Subjects  Review 
protocol  submitted  to  the  Air  Force  Research  Laboratory,  Human  Subjects  Review  Board 
located  at  Wright  Patterson  Air  Force  Base.  Full  documentation  of  research  processes 
and  procedures  were  documented  and  provided  to  increase  the  reliability  of  this  study. 
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Population 

The  population  of  this  research  is  all  users  of  AFKN  CoPs  from  August  until 
October,  2004. 

Limitations 

Due  to  the  nature  of  Case  Study  research,  sample  size  is  a  limitation  of  this 
research.  Another  limitation  may  be  the  lack  of  negative  responses  from  users.  This  is  a 
limitation  due  to  the  nature  of  research  questions  2b  and  3b  in  identifying  factors  that 
discouraged  use  of  the  CoPs. 

Data  Collection 

The  steps  taken  to  ensure  the  reliability  and  validity  of  the  case  study  research 
guided  the  data  collection  process.  This  research  used  semi-structured  interviews  to 
gather  data  from  CoP  users.  Interviews  were  conducted  on  a  voluntary  basis,  and  CoPs 
were  randomly  selected  to  participate  in  the  interview.  To  conduct  the  interviews,  the 
researcher  contacted  individual  CoP  administrators  and  requested  their  assistance  in 
identifying  three  to  four  users  who  would  assist  in  the  interview.  After  the  users  were 
contacted,  they  were  sent  an  electronic  copy  of  the  survey  that  included  demographic 
data.  The  users  then  completed  the  survey  and  returned  it  to  the  researcher  prior  to 
performing  the  interview.  To  remain  consistent  between  the  interviewees  that  were  in  the 
local  area  and  the  ones  who  were  located  elsewhere,  all  interviews  were  conducted  over 
the  phone. 

Question  Development 

The  survey  questions  were  taken  directly  from  the  Venkatesh  (2003)  UTAUT 
research,  with  the  exception  of  question  25.  Question  25  was  added  to  elicit  the 


35 


perception  of  peers  in  regards  to  using  the  system.  The  interview  consisted  of  10 
questions  that  dealt  with  one  of  six  constructs: 

•  Perceived  Usefulness 

•  Ease  of  Use 

•  Social  influence 

•  Facilitating  conditions 

•  Self-efficacy/Anxiety 

•  Behavioral  intention  to  use  the  system 

Additionally,  two  general  questions  were  asked  to  provide  the  interviewee  a 
chance  to  voice  their  overall  impression  on  the  CoPs.  The  demographic  information  was 
designed  to  identify: 

•  Position  in  the  CoP  (User,  Administrators,  etc.) 

•  The  access  to  the  CoP  (open  or  closed) 

•  How  the  CoP  is  organized  (team,  function,  directorates) 

•  If  the  system  was  mandatory  or  optional  to  use 

•  The  length  of  time  the  user  had  been  subscribed/registered  with  the  CoP 

•  How  many  times  (sessions)  the  user  accessed  the  system  during  a  given 
week 

•  How  long  they  spent  on  the  system  during  the  sessions. 

The  survey  is  located  in  Appendix  B  and  the  interview  is  located  in  Appendix  C. 

Pilot  Study 

A  pilot  study  was  performed  with  five  graduate  students  with  experience  using 
CoPs,  to  validate  the  instrument.  Based  on  this  study  there  were  several  changes 
incorporated  into  the  interview.  Two  questions  were  added  to  the  demographics  section 
to  identify  CoP  usage  as  well  as  specific  areas  within  the  CoP  that  the  individuals 
interviewed  are  using.  The  pre-interview  questions  derived  from  Venkatesh  et  al.  (2003) 
were  unchanged;  however,  question  25  (“My  peers  support  using  the  system”)  was  added  to 
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ascertain  the  perceptions  of  peers,  which  wasn’t  included  in  Venkatesh’s  original  survey. 
The  interview  questions  were  changed  slightly  to  impove  clarity. 

Interview  Procedures 

Participants  were  scheduled  for  an  interview  at  a  time  that  was  convenient  to 
them.  Prior  to  the  interview,  each  participant  was  sent  electronically  a  copy  of  the  survey 
with  the  demographic  information  on  it.  This  was  then  filled  out  by  the  participant  and 
returned  to  the  researcher  prior  to  the  start  of  the  interview.  At  the  start  of  the  interview, 
each  participant  was  asked  whether  or  not  they  would  consent  to  the  interview  being 
audio  taped  which  aids  in  the  construction  of  the  transcripts.  After  the  interview  was 
complete,  the  interviewer  created  a  transcript  of  the  interview  and  sent  it  to  the 
participant  for  review.  At  that  time,  each  participant  was  given  an  informed  consent 
letter  that  included  the  consent  to  be  quoted.  The  participants  then  reviewed  the 
transcripts,  and  then  electronically  signed  the  informed  consent  letter  and  returned  the 
letter  with  any  modifications  that  they  made  to  the  transcripts. 

Data  Analysis 

Interview  transcripts  provided  the  basis  for  the  majority  of  the  data  analysis  in  this 
research.  Previous  research  on  CoPs  provided  additional  support  to  the  data  analysis. 

Two  techniques  were  used  to  analyze  the  interview  transcripts:  1)  comparison  of  results 
to  theory  in  the  literature,  and  2)  pattern  matching. 

Pattern  Matching 

A  comparison  of  empirically  based  patterns  with  a  predicted  one  describes  pattern 
matching.  (Yin,  2003)  Patterns  that  emerged  from  the  interviews  were  compared  to  the 
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other  interviews  and  to  written  documentation.  The  resulting  similarities  were  identified 
in  the  analysis. 

Comparison  of  Results  to  Theory 

As  previously  stated,  the  purpose  of  this  research  was  to  identify  use  and 
acceptance  of  AFKN  CoPs.  The  results  of  the  survey  and  interviews  were  compared 
against  previous  TAM  findings. 

Chapter  Overview 

This  chapter  described  the  data  analysis,  data  collection,  research  design,  and 
quality  issues;  and  presented  the  methodology  used  in  conducting  this  research  project. 
The  methodology  was  broken  into  three  main  sections:  the  overall  research  method 
selection  process,  the  method  for  evaluating  the  quantitative  portion  of  the  research,  and 
finally  the  design  of  the  qualitative  portion  of  the  study.  The  next  chapter  will  presents 
results  and  analysis  of  the  data. 
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IV.  Results  and  Analysis 


Overview 

This  chapter  presents  the  results  of  the  research  study.  These  results  will  be 
presented  and  analyzed  in  four  distinct  sections: 

1 . )  The  data  from  the  Fitzgerald  &  Hinrichsen  (2004)  study  will  be  presented 

with  the  blocking  factors  of  the  CoP’s  functional  organization,  formality, 
access  (open  or  closed),  grade/position,  and  length  of  time  using  CoPs.  These 
results  will  be  used  to  answer  research  questions  la  through  le. 

2. )  The  results  from  the  pre-interview  survey  will  be  presented  along  with  the 

current  research  model.  These  results  will  be  used  to  evaluate  the  research 
model  and  to  assist  in  answering  section  three. 

3. )  The  factors  derived  from  the  21  interviews  will  be  presented.  These  results 

will  be  used  to  answer  research  questions  2a  through  4. 

4. )  The  results  to  all  research  questions  will  be  discussed. 

Results  to  Research  Questions  la  through  le 

To  answer  questions  la  through  le,  the  raw  data  from  the  Fitzgerald  and 
Hinrichsen  (2004)  studies  was  re-examined  by  blocking  for  the  factors  of  functional 
makeup,  formality,  access,  length  of  time  being  associated  with  CoPs  and  user’s 
grade/position.  See  the  quantitative  research  design  section  of  chapter  three  for  a  detailed 
breakdown  of  the  survey  methodology  and  response  rate  for  this  data.  These  factors  were 
acquired  from  information  on  the  CoP  or  from  data  that  was  already  available  in  the 
dataset.  If  the  information  was  not  available  (primarily  due  to  the  CoP  being  unavailable) 
then  the  response  was  coded  as  “unknown”  and  was  not  factored  into  analysis.  Of  the 
1042  usable  surveys,  136  fell  into  the  category  of  unknown.  Of  the  42  questions  asked  in 
the  survey,  only  30  were  used  for  the  actual  study.  See  Table  7  for  a  breakdown  of  the 
corresponding  construct  and  question. 
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Construct 

Question(s) 

Absence  of  "not 
invented  here" 
syndrome 

My  CoP  encourages  its  members  to  use  materials  originating  outside  our 
CoP. 

Adaptive 

My  fellow  CoP  members  try  new  tools  or  suggestions. 

Anonymity 

1  would  participate  more  often  in  my  CoP  if  1  could  remain  anonymous. 

Anonymity 

1  would  share  my  opinions  and  insights  more  often  in  my  CoP  if  1  could 
remain  anonymous. 

Curious 

Members  of  my  CoP  are  eager  to  learn  new  things. 

Facilitating  Conditions 

Training  in  the  use  of  my  CoP  was  available  to  me. 

Facilitating  Conditions 

1  have  the  knowledge  necessary  to  use  my  CoP. 

Facilitator 

The  efforts  of  my  CoP's  facilitator  affect  how  much  1  participate  within  my 
CoP. 

Information  sharing 

Information  is  shared  in  my  CoP. 

Job  Fit 

Use  of  CoPs  can  significantly  increase  the  quality  of  output  on  my  job. 

Job  Fit 

Use  of  CoPs  will  affect  the  performance  of  my  job. 

Knowledge  Champion 

A  knowledge  champion  is  responsible  for  invigorating  a  CoP,  encouraging 
CoP  members  to  participate  and  share  knowledge,  highlighting  successes, 
recognizing  the  contributions  of  members,  and  so  on:  my  CoP  has  a 
knowledge  champion. 

Outcome  Expectations 

If  1  use  my  CoP  1  will  increase  my  chances  of  obtaining  a  promotion. 

Outcome  Expectations 

If  1  use  my  CoP  1  will  increase  my  effectiveness  on  the  job. 

Positive  culture 

Members  of  my  CoP  work  to  accomplish  common  goals. 

Reuse 

My  CoP  ensures  members  know  where  to  find  resources. 

Rewards 

My  CoP  recognizes  or  rewards  its  members  for  making  contributions. 

Security  Constraints 

The  level  of  security  my  job  deals  with  limits  my  ability  to  use  CoPs  in  my 
work. 

Security  Constraints 

1  would  participate  more  in  my  CoP  if  the  sharing  of  classified  and  higher 
information  was  allowed. 

Social  Factors 

My  supervisor  is  very  supportive  of  my  use  of  CoPs  in  my  job. 

Social  Factors 

In  general,  my  organization  has  supported  my  use  of  CoPs. 

Strong  culture 

Most  members  of  my  CoP  agree  on  major  issues  discussed  in  our 
community. 

Teamwork 

Teamwork  is  valued  in  my  CoP. 

Technically  minded 

Members  of  my  CoP  are  technically  competent  enough  to  use  our  CoP. 

Tolerance 

In  order  for  a  CoP  to  thrive,  members  must  understand  that  it  is  okay  to 
make  mistakes:  my  fellow  CoP  members  are  patient  with  people  who  make 
honest  mistakes. 

Trust 

1  trust  my  fellow  CoP  members. 

Trust 

Information  obtained  from  my  CoP  is  reliable  enough  to  use  in  my  job. 

Trust 

The  members  of  my  CoP  are  competent  enough  in  their  job  knowledge  to 
provide  accurate  information  to  others  within  the  CoP. 

Willingness  to  Share 

Sharing  my  job  knowledge  with  other  members  of  my  CoP  will  make  me 
more  valuable  to  my  organization. 

Willingness  to  Share 

1  have  no  reservations  about  sharing  my  job  knowledge  with  other 
members  of  my  CoP. 

Table  7  -  Survey  Questions  and  Constructs  (Fitzgerald,  2004;  Hinrichsen,  2004) 


Results  derived  from  the  data  analysis  of  this  data  will  be  presented  with  the 
applicable  research  questions.  Please  refer  to  Table  7  in  regards  to  the  specific 
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question/construct.  An  “H”  indicates  that  the  mean  of  the  responses  for  that  particular 

area  was  significantly  higher  (a  =  0.05)  than  those  with  an  “L.”  A  block  that  is  blank 

indicates  a  returned  mean  that  was  not  statistically  different  from  the  other  responses. 

la.  Based  on  existing  models,  is  there  a  difference  between  factors  based  on  whether 
the  CoP  is  used  by  teams,  function,  or  directorates? 


Percentages 

19 

69 

12 

Construct 

Team 

Function 

Directorate 

Trust 

H 

L 

L 

Information  Sharing 

H 

L 

L 

Anonymity 

L 

H 

Tolerance 

H 

L 

Facilitating 

Conditions 

H 

H 

L 

Table  8  -  Findings  Based  on  Teams,  Functions,  and  Directorates 


Based  on  findings  in  Table  8  above,  there  is  a  difference  between  the  five 
constructs  listed  for  CoPs  used  by  teams,  functions,  and  directorates.  The  identified 
constructs  will  be  described  below. 

Trust 


Teams  tend  to  be  more  trusting  of  others  in  their  CoP  and  the  infonnation  that  is 
available  on  their  CoPs.  This  could  be  attributed  to  the  inherently  small  size  and  close- 
knit  relationship  that  teams  tend  to  have.  In  contrast,  functions,  which  are  traditionally 
more  geographically  separated  and  have  a  lower  personal  interface  for  that  reason,  have  a 
lower  perception  of  the  reliability  of  the  data  on  their  CoPs.  Additionally  directorates, 
which  tend  to  be  larger  and  more  formally  structured  than  teams,  also  share  this  low 
perception  of  the  data’s  reliability  on  their  CoP. 
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Information  Sharing 

Again,  the  larger  and  more  dispersed  functions  and  directorates  have  a  lower 
perception  of  information  sharing  than  the  smaller  and  cohesive  teams.  This  could  be 
associated  to  lack  of  knowledge  of  how  others  are  using  the  system  or  a  poor 
understanding  of  exactly  how  others  are  sharing.  It  is  also  interesting  to  note  that  the 
results  of  trust  coincide  with  the  results  of  information  sharing.  This  might  imply  that  a 
level  of  trust  is  involved  with  the  sharing  of  infonnation. 

Anonymity 

This  is  not  as  much  of  a  factor  in  participation  for  teams,  as  it  is  in  directorates. 
This  could  be  attributed  again  to  the  close-knit  nature  of  teams  in  which  even  if  the  user 
had  anonymity,  the  other  team  members  would  probably  know  who  the  user  was  anyway. 
In  the  more  structured  directorate  setting,  some  users  may  be  more  likely  to  share  their 
opinions  if  they  did  not  have  to  fear  some  form  of  retribution  for  giving  their  input. 

Again,  these  results  directly  coincide,  although  inversely  due  to  the  nature  of  the 
question,  to  trust. 

Tolerance 

Teams  have  a  higher  tolerance  of  those  who  make  mistakes  than  directorates.  It  is 
interesting  that  tolerance  is  different  between  any  of  the  respective  areas.  The  idea  that 
an  individual  in  a  larger,  more  formalized  setting  would  not  be  likely  to  make  a  mistake 
and  as  such,  the  other  members  might  would  be  less  patient  with  them  seems 
inconsistent. 
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Facilitating  Conditions 

Members  of  functions  and  teams  have  a  higher  degree  of  confidence  that  they 

have  the  knowledge  necessary  to  use  their  CoPs,  over  the  members  of  directorates.  It 

would  be  assumed  that  a  directorate  would  have  the  resources  available  for  their  users  in 

the  event  that  they  would  need  assistance;  where  the  team  or  functional  should  have  less 

resources  and  would  therefore  not  have  a  defined  infrastructure  available  to  support  the 

users  in  the  event  that  they  needed  assistance.  Upon  looking  deeper  into  this  response 

though,  it  can  be  seen  how  individuals  would  be  less  apt  to  seek  out  assistance  in  a 

mandated  system,  especially  one  that  is  as  informally  defined  as  CoPs  are. 

lb.  Based  on  existing  models,  is  there  a  difference  between  factors  based  on  whether 
the  CoP  is  formed  mandatory  or  optional  (informal  or  formal)? 


Percentages 

87 

13 

Construct 

Informal 

Formal 

Facilitating  Conditions 

H 

L 

Table  9  -  Findings  Based  on  Mandatory  or  Optional  Participation 

Based  on  findings  in  Table  9  above,  there  is  a  difference  between  the  facilitating 
condition  construct  for  CoPs  that  are  informally  and  formally  managed.  The  identified 
construct  will  be  described  below. 

Facilitating  Conditions 

This  was  the  only  difference  that  stood  out  between  infonnal  and  formal  CoPs. 
This  response  does  reflect  those  found  when  looking  at  teams  functions  and  directorates 
due  to  the  formal  nature  of  a  directorate  over  the  informal  nature  of  team  and  functional 
CoPs. 

lc.  Based  on  existing  models,  is  there  a  difference  between  factors  based  on  whether 
the  CoP  is  open  or  closed? 
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Table  10  -  Findings  Based  on  Open  and  Closed  Access 

Based  on  findings  in  Table  10  above,  there  is  a  difference  between  the  three 
constructs  listed  for  CoPs  that  have  open  or  closed  access.  The  identified  constructs  will 
be  described  below. 

Security  Constraints 

Members  of  open  CoPs,  generally  have  a  lower  perception  of  the  security 
constraints  in  place  for  their  CoP  over  those  that  belong  to  closed  CoPs.  The  fact  that 
individuals  participate  in  a  CoP  that  does  or  does  not  restrict  access,  signifies  their  level 
of  concern  for  security  constraints. 

Social  Factors 

Individuals  in  closed  CoPs  tend  to  have  a  lower  perception  of  the  social  factors 
than  those  who  belong  to  open  CoPs.  That  is  to  say,  that  members  in  open  CoPs  feel  that 
their  superiors  as  well  as  their  organization  as  a  whole  are  more  supportive  in  their  use  of 
the  CoPs.  This  finding  does  not  lend  itself  to  a  simple  explanation. 

Willingness  to  Share 

Similarly  to  social  factors,  the  perception  of  a  CoP’s  willingness  to  share  is  higher 
in  open  CoPs  than  in  closed  CoPs.  It  might  have  been  expected  that  an  individual  would 
be  more  apt  to  share  with  individuals  in  a  controlled  CoP  because  they  would  know 
exactly  who  would  be  accessing  the  shared  infonnation.  Although  by  nature,  an 
individual  that  participates  in  an  open  CoP  might  have  a  higher  level  of  trust,  but  the 
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construct  of  trust  was  not  supported  as  being  significantly  different  between  open  and 
closed  CoPs. 
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Id.  Based  on  existing  models,  is  there  a  difference  between  factors  based  on  how  long  the  individual  has  been  with  the 
CoP? 
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Table  11  -  Findings  Based  on  Length  of  Time  Using  CoPs 


Based  on  the  findings  in  Table  11,  there  is  a  difference  between  the  17  constructs  listed  based  on  the  length  of  time  that 


members  have  belonged  to  a  CoP.  The  identified  constructs  will  be  described  below. 


In  evaluating  the  constructs  against  time,  each  of  the  different  levels  were 
evaluated  individually  against  the  other  levels.  The  levels  were  people  that  had  used  the 
system:  for  less  than  a  month  (Ltl),  from  1  to  12  months  (1-12),  from  13  to  24  months 
(13-24),  from  25  to  36  months  (25-36),  and  more  than  36  months  (MT  36).  The  results  of 
this  analysis  can  be  seen  in  Table  1 1  under  the  “Time  using  CoPs”  heading. 

Additionally,  although  some  of  the  results  were  not  statistically  significant,  the  means  of 
the  responses  displayed  a  trend  over  time;  therefore,  the  column  “order”  was  created  to 
display  the  direction  of  this  trend  over  time.  After  the  initial  evaluation,  dummy 
variables  were  created  for  users  who  had  used  the  system  for  more  than  12  months  versus 
those  that  had  used  the  system  up  to  and  including  12  months.  The  results  to  this  analysis 
can  be  found  under  the  “Time  12  Mo”  heading  in  Table  11.  Another  dummy  variable 
was  created  for  users  who  had  used  the  system  for  more  than  24  months  against  those 
who  had  used  it  up  to  and  including  24  months.  The  results  to  this  analysis  can  be  found 
under  the  “Time  24  Mo”  heading  in  Table  11. 

Due  to  the  extensive  nature  of  the  findings  of  the  variable  of  time,  in  order  to 
answer  research  question  Id  the  differences  and  similarities  between  the  variables  will  be 
presented  and  analyzed.  All  of  the  variables  with  the  exception  of  Anonymity  and 
Security  Constraints  showed  a  decline  over  time.  Specifically,  seven  of  the  constructs 
incrementally  decreased  as  the  individuals  had  been  using  the  system  over  time.  Nine  of 
the  constructs  showed  a  significant  difference  at  the  12-month  point,  while  another  six 
showed  a  distinct  difference  at  the  24-month  point.  Only  Strong  Culture  did  not  show  a 
difference  at  the  24-month  point  where  it  did  at  the  12-month  point.  Although  the 
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Curious  construct  did  not  have  any  significant  finding  between  the  different  levels,  the 
means  of  the  responses  consistently  decreased  from  a  high  for  those  using  the  system  for 
less  than  a  month  to  a  low  for  users  of  the  system  who  had  been  with  the  CoP  for  more 
than  36  months.  Finally,  Adaptive  did  not  show  any  significant  differences  at  the  12  and 
24-month  points,  but  it  did  show  a  significant  difference  between  users  who  had  been  on 
the  CoP  for  less  than  one  month  and  those  who  had  been  using  the  system  for  over  36 
months. 
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Table  12  -  Findings  Based  on  Grade/Position  of  CoP  User 


Based  on  the  findings  in  Table  12,  there  is  a  difference  between  the  1 1  constructs  listed  for  members  of  CoPs  based  on 


their  rank/position.  The  identified  constructs  will  be  described  below. 


In  evaluating  the  data  to  answer  research  question  le,  all  constructs  were  assessed 
against  the  1 1  different  available  variables.  After  the  initial  assessment,  dummy  variables 
were  created  to  evaluate  differences  based  on  rank/position  based  on  the  respondents 
status  as  being  either  enlisted,  officer,  civilian,  contractor,  or  other.  It  is  unknown  what 
constitutes  other;  however,  because  there  were  respondents  that  selected  other  it  was 
included.  Next,  a  dummy  variable  between  military  responses  and  civilians,  to  include 
contractors  and  others,  was  created.  Finally,  a  subjective  variable  was  created  based  on 
position.  Gsl-gs5  and  el-e4  were  combined  to  represent  low;  e5-e6,  ol-o3,  and  gs6-gsl0 
were  combined  to  represent  mid;  e7-e9,  04-o6,  o7-ol0,  and  gs  1 1  -gs  1 5  were  combined  to 
represent  high.  Contractor  was  combined  with  other  due  to  the  lack  of  delineation  within 
the  contractor  variable. 

As  was  the  case  in  answering  research  question  Id,  differences  and  similarities 
between  the  variables  of  grade  and  position  will  be  presented  and  analyzed.  In  looking 
through  the  responses,  e5-e6  tends  to  answer  higher  in  general,  where  04-06s  tend  to 
answer  lower  than  others  do.  No  specific  grade  stands  out  as  being  consistently  different 
from  the  others  across  all  of  the  variables.  When  the  specific  grades  are  blocked  out  by 
their  respective  category  (officer,  enlisted,  contractor,  or  civilian)  some  significant 
differences  emerge.  In  general,  officers  responses  tend  to  be  lower,  specifically  when 
compared  to  the  responses  of  enlisted  personnel.  When  looking  at  the  grade  block 
between  military  and  civilians,  military  has  a  higher  expectation  in  regards  to  outcome 
expectations  and  security  constraints,  where  civilians  have  a  higher  perception  of  the 
facilitating  conditions  and  social  factors  impact.  Finally,  overall  the  high-level 
respondents  answered  lower  than  the  mid,  and  low-level  users. 
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Results  of  the  pre-interview  survey 

As  stated  previously,  a  28  question  pre-interview  survey  was  given  to  the 
individuals  participating  in  the  interview.  The  instrument’s  questions  used  in  this  survey 
were  directly  derived  from  Venkatesh  et  al.  (2003).  The  results  of  this  survey  were 
analyzed  using  the  JMP  Statistical  Discovery  Software  version  5.0.1,  in  order  to 
test/validate  the  research  model  presented  in  Chapter  two  (See  Figure  13).  Of  the  21 
individuals  interviewed  during  this  research,  only  20  completed  the  survey.  The  21st 
person’s  interview  was  found  to  be  incomplete  after  all  data  had  been  collected  and  the 
individual  was  unavailable  to  complete  the  survey  at  that  point  due  to  a  deployment. 

The  responses  were  evaluated  to  determine  how  predictive  each  construct  was  in 
regards  to  the  various  constructs  within  the  model.  See  Figure  17  for  the  current  research 
model  with  the  calculated  adjusted  R"  results.  The  values  that  are  next  to  the  arrows 
relate  to  the  individual  predictability,  while  the  values  inside  of  the  boxes  correlate  to  a 
combination  of  all  inputs  to  that  particular  box. 


Figure  17  -  Current  Research  Model  Results 


The  model  supports  previous  finding  in  regards  to  the  affects  that  user  acceptance 
enablers  have  on  use  and  ease  of  use,  as  well  as  the  affect  that  ease  of  use  has  on  use. 
There  is  also  very  strong  support  for  ease  of  use  and  user  acceptance  enabler’s  affect  on 
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use.  Individually,  ease  of  use,  use,  and  social  influences  do  not  predict  intention  to  use 
well;  however,  these  three  constructs  combined  do  a  relatively  good  job  in  predicting 
intention  to  use.  Finally,  intention  to  use  and  facilitating  conditions  do  a  poor  job  of 
predicting  usage  behavior,  but  the  two  constructs  combined  do  a  relatively  good  job  of 
predicting  usage  behavior. 

After  evaluating  the  model,  each  variable  was  evaluated  for  predictability  against 
the  other  variables.  See  Table  13  below  for  summary  of  the  findings.  Findings  are 

reported  as  adjusted  R  with  an  d  =  0.05.  The  results  show  the  variable  on  the  left’s 


ability  to  predict  the  variable  listed  on  the  top. 


Variable 

UAE 

EU 

U 

S 

1 

FC 

UB 

User  Acceptance 
Enablers  (UAE) 

X 

0.895 

0.717 

0.422 

0.797 

0.693 

0.431 

Ease  of  Use  (EU) 

0.0342 

X 

0.731 

0.322 

0.37 

0.489 

0.183 

Perceived 
Usefulness  (U) 

0.225 

0.633 

X 

0.333 

-0.014 

0.354 

0.114 

Social 

Influences  (S) 

-0.289 

0.495 

0.463 

X 

0.099 

0.644 

0.075 

Intention  to 

Use  (1) 

0.219 

0.274 

-0.034 

0.099 

X 

0.412 

0.092 

Facilitation 
Conditions  (FC) 

0.018 

0.51 

0.067 

0.52 

0.482 

X 

-0.014 

Usage 

Behavior  (UB) 

0.079 

-0.057 

0.06 

-0.051 

-0.025 

-0.045 

X 

Table  13  -  Results  of  Individual  Variables 


An  interesting  finding  from  this  analysis  is  the  amount  of  predictability  that  user 
acceptance  enabler  has  on  all  the  variables.  Additionally,  the  construct  of  facilitating 
conditions  was  predictive  of  ease  of  use,  social  influences,  and  intention  to  use,  while 
social  influences  was  somewhat  predictive  of  ease  of  use,  perceived  usefulness,  and 
facilitating  conditions.  Based  on  these  findings,  the  variables  were  combined  and  the 
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2 

adjusted  R  was  evaluated  (see  Table  14)  to  ascertain  any  relationships  that  may  help 
identify  stronger  interaction  within  the  constructs. 


By  Predicted 

UAE+FC+S 

UB 

0.740 

UAE+S 

UB 

0.660 

UAE+FC 

UB 

0.640 

EU+U+I 

UB 

0.445 

U+l 

UB 

0.382 

EU+U 

UB 

0.267 

FC+S 

UB 

0.062 

EU+I 

UB 

-0.194 

FC+S 

UAE 

0.339 

UAE+S 

U 

0.827 

UAE+FC 

U 

0.813 

FC+S 

U 

0.517 

EU+S 

1 

0.595 

EU+U 

1 

0.473 

FC+S 

1 

0.305 

U+S 

1 

-0.080 

UAE+S 

EU 

0.768 

UAE+FC 

EU 

0.492 

FC+S 

EU 

0.331 

Highest  to  Lowest 

UAE+S 

U 

0.827 

UAE+FC 

U 

0.813 

UAE+S 

EU 

0.768 

UAE+FC+S 

UB 

0.740 

UAE+S 

UB 

0.660 

UAE+FC 

UB 

0.640 

EU+S 

1 

0.595 

FC+S 

U 

0.517 

UAE+FC 

EU 

0.492 

EU+U 

1 

0.473 

EU+U+I 

UB 

0.445 

U+l 

UB 

0.382 

FC+S 

UAE 

0.339 

FC+S 

EU 

0.331 

FC+S 

1 

0.305 

EU+U 

UB 

0.267 

FC+S 

UB 

0.062 

U+S 

1 

-0.080 

EU+I 

UB 

-0.194 

Table  14  -  Ad-Hoe  Data  Analysis  Results 


As  noted  in  the  previous  section,  user  acceptances  enablers  continues  to  have  a 
significant  impact  on  the  model.  Additionally,  facilitating  conditions  and  social 
influences  provide  a  much  greater  influence  when  evaluated  earlier  in  the  model, 
specifically  prior  to  ease  of  use  and  perceived  usefulness.  The  issue  that  arises  from  this 
analysis  is  the  disconnect  between  ease  of  use,  perceived  usefulness,  and  intention  to  use 
in  predicting  usage  behavior.  This  notion  was  inferred  by  both  Dasgupta  et  al.  (2002) 
and  Fitzgerald  (2004)  when  they  were  evaluating  predictability  of  usage  in  a 
collaborative  technology  and  CoP  respectively.  Based  on  the  findings  from  the  initial 
evaluation  as  well  as  the  ad-hoc  analysis,  the  initial  research  model  has  been  altered  in 
order  to  better  predict  usage  behavior,  see  Figure  18  for  the  modified  research  model. 
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Figure  18  -  Modified  Research  Model 


Results  to  Research  Questions  2a  through  4 

To  review,  the  information  in  the  following  section  was  obtained  through  one-on- 
one  interviews  conducted  from  10  October  through  30  November  2004.  During  this 
time,  2 1  individuals  were  interviewed  based  on  the  interview  questions  found  in 
Appendix  C.  The  results  were  compiled  and  categorized  based  on  the  nature  of  the 
response.  See  Appendix  F  for  the  complete  findings  matrix.  The  responses  were  broken 
down  by  direct-question  responses  and  open-ended  responses.  The  positive  and  negative 
responses  to  the  direct-questions  are  presented  in  Table  15  and  Table  16. 
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Direct  question  responses  of  users  who  had  used  the  CoP  for  8  months  or  less 


Answers  to  direct  questions 

Used  CoP  8  months  or  less 

Neg 

Pos 

Received  actual  training  on  system 

N 

N 

N 

N 

N 

N 

N 

N 

N 

N 

N 

11 

0 

Co-workers  support  CoPs 

P 

P 

P 

P 

P 

N 

P 

N 

N 

P 

P 

N 

4 

8 

Boss/superiors  support  using  CoPs 

P 

P 

P 

P 

P 

N 

N 

N 

N 

P 

P 

N 

5 

7 

Feel  comfortable  using  the  system 

P 

P 

P 

P 

P 

N 

P 

N 

N 

P 

P 

P 

3 

9 

Intend  to  keep  using  the  systems 

P 

P 

P 

N 

P 

P 

P 

P 

P 

1 

8 

Easy  to  use/learn 

N 

P 

P 

P 

P 

N 

P 

N 

P 

P 

P 

P 

3 

9 

If  you  need  help,  where  do  you  go? 

Use  AFKN  administrators 

P 

P 

P 

0 

3 

Use  CoP  administrators 

P 

P 

P 

P 

0 

4 

Use  co-worker  for  assistant 

P 

□ 

P 

0 

3 

Use  help  section  on  CoP/AFKN 

N 

N 

N 

□ 

■ 

N 

5 

0 

What  kind  of  help  function  would  you  like? 

Would  like  a  good  users  guide 

P 

■ 

■ 

0 

1 

Would  like  AFKN  Help  Desk 

P 

0 

1 

Would  like  FAQ  (help  function) 

P 

□ 

□ 

B 

0 

4 

Would  like  interactive  video  (CBT) 

□ 

□ 

0 

3 

Would  like  to  contact  AFKN  administrators 

P 

□ 

□ 

□ 

0 

4 

Would  like  to  contact  CoP  administrators 

P 

0 

P 

0 

3 

Table  15  -  Direct  question  responses:  CoP  users  less  than  or  equal  to  eight  months 


1 1  of  the  users  reported  not  receiving  any  kind  of  training  on  the  system.  Eight  of 
the  users  felt  that  their  co-workers  supported  the  use  of  the  system,  and  seven  felt  that 
their  bosses  supported  using  the  system.  Nine  users  felt  comfortable  using  the  system 
where  eight  of  the  users  that  responded  in  regards  to  continued  use  felt  that  they  would 
continue  using  the  system.  Nine  of  the  users  felt  the  system  was  easy  to  learn.  In 
general,  the  majority  of  respondents  felt  supported  in  their  use  of  CoPs  and  felt  that  it  is  a 
good  system  that  was  easy  to  learn  and  that  they  would  continue  to  use  it.  The  only 
negative  aspect  was  the  lack  of  training  received  by  the  users. 

When  users  were  having  problems  with  their  CoP,  they  tended  to  rely  on  AFKN 
or  CoP  administrators,  or  they  asked  a  co-worker  for  assistance.  Five  of  the  respondents 
reported  having  trouble  with  the  online  help,  or  said  that  they  had  not  looked  for  it. 
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As  far  as  what  users  would  like  to  see  in  regards  to  a  help  function,  only  one 
person  said  that  they  would  like  some  kind  of  user’s  manual.  Many  of  the  users  said  that 
they  would  like  some  kind  of  computer-based  training  (CBT)  or  interactive  video  to 
demonstrate  the  features  of  AFKN  CoPs.  Also,  in  regards  to  needing  help  with  CoPs, 
most  would  begin  by  either  asking  a  co-worker  if  available  or  they  would  like  to  have 
some  frequently  asked  questions  (FAQ)  section  that  they  could  access.  They  would  next 
like  to  contact  their  CoP  administrator  or  an  AFKN  administrator  either  by  e-mail  or  by 
phone.  Several  individuals  mentioned  the  need  to  speak  to  a  person,  specifically  through 
some  form  of  help  desk. 

Direct  question  responses  of  users  who  had  used  the  CoP  for  12  months  or  more 


Answers  to  direct  questions 

Used  CoP  1  year  or  more 

Neg 

Pos 

Received  actual  training  on  system 

N 

P 

N 

N 

N 

N 

N 

P 

6 

2 

Co-workers  support  CoPs 

N 

N 

P 

N 

P 

N 

N 

6 

2 

Boss/superiors  support  using  CoPs 

N 

P 

N 

N 

N 

P 

P 

N 

N 

6 

3 

Feel  comfortable  using  the  system 

N 

P 

P 

P 

P 

P 

P 

1 

6 

Intend  to  keep  using  the  systems 

P 

P 

P 

0 

3 

Easy  to  use/learn 

P 

P 

P 

P 

P 

P 

P 

P 

N 

1 

8 

If  you  need  help,  where  do  you  go? 

Use  AFKN  administrators 

P 

P 

0 

2 

Use  CoP  administrators 

P 

P 

P 

P 

P 

0 

5 

Use  co-worker  for  assistant 

P 

P 

0 

2 

Use  help  section  on  CoP/AFKN 

P 

P 

N 

P 

1 

3 

What  kind  of  help  function  would  you  like? 

Would  like  a  good  users  guide 

0 

0 

Would  like  AFKN  Help  Desk 

P 

P 

P 

0 

3 

Would  like  FAQ  (help  function) 

N 

P 

P 

P 

P 

P 

P 

1 

6 

Would  like  interactive  video  (CBT) 

P 

0 

1 

Would  like  to  contact  AFKN  administrators 

P 

P 

P 

0 

3 

Would  like  to  contact  CoP  administrators 

P 

P 

P 

P 

0 

4 

Table  16  -  Direct  question  responses:  CoP  users  greater  than  or  equal  to  12  months 


Only  two  users  (out  of  all  2 1  interviewees)  reported  receiving  any  form  of 
training.  Although  one  CoP  administrator  that  was  interviewed  did  say  that  he  does 
provide  one-on-one  training  to  new  users.  Out  of  the  users  who  had  been  using  the 
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system  for  more  than  a  year,  only  two  of  these  individuals  mentioned  support  from  their 
co-workers  and  only  three  of  them  recognized  that  their  bosses  support  using  it.  Only 
one  user  did  not  feel  comfortable  using  the  system,  and  this  individual  felt  that  was 
because  he  was  only  using  a  small  section  of  the  CoP  and  did  not  have  a  thorough 
knowledge  of  the  other  sections  of  the  system.  No  user  felt  that  they  would  want  to  stop 
using  the  system  in  lieu  of  another  method  of  reaching  similar  objectives,  although  most 
of  the  users  interviewed  could  not  think  of  a  system  that  could  replace  some  of  the 
functions  that  the  CoP  provides  to  them.  In  addition,  the  majority  of  the  users  felt  that 
overall  the  system  was  easy  to  learn. 

For  help,  the  users  who  had  been  using  their  CoP  for  over  a  year  would  primarily 
rely  on  their  CoP’s  administrator.  Some  would  rely  on  a  co-worker  or  an  AFKN 
administrator.  Only  three  users  felt  that  they  would  want  to  use  the  help  section  on  the 
CoP. 

Users  who  had  been  using  the  system  for  more  than  a  year  overwhelmingly  would 
like  to  see  some  form  of  FAQ  help  function.  Many  of  those  interviewed  would  use  the 
FAQ  first  and  then  would  either  like  to  contact  their  administrator  or  the  AFKN 
administrator  either  directly  or  through  some  form  of  AFKN  help  desk. 

Next,  the  open-ended  question  responses  will  be  presented  separately  based  on  the 
particular  research  question  that  they  pertain  to.  Both  the  direct-questions  and  open- 
ended  questions  will  be  presented  separately  based  on  whether  the  user  who  responded 
had  belonged  to  a  CoP  eight  months  or  less  or  for  a  year  or  more.  Of  the  2 1  individuals 
interviewed,  12  had  used  the  system  for  eight  months  or  less,  while  the  other  nine 
individuals  had  used  the  system  for  a  year  or  more.  The  key  concepts  discovered  from 
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the  responses  to  the  open-ended  questions  were  grouped  into  eight  categories  (See  Table 
17). 


Document  Management 

Interface  and  Usability 

Forums 

Records  Management 

Connectivity 

Training 

Usage  Issues 

Facilitator 

Table  17  -  Main  Categories  of  Interview  Response  Key  Concepts 


2a.  What  are  the  specific  factors  that  encouraged  an  individual  to  participate  in  a 
particular  CoP  when  initially  starting  to  use  the  CoP? 

Document  Management 

Three  items  stood  out  as  being  positive  in  this  main  heading:  posting  and  sharing 
documents  and  forms,  having  the  data  in  one  place,  and  using  the  CoP  as  a  reference 
and/or  authoritative  source  of  data.  The  need  to  post  and  share  documents  was  the  most 
noted  factor  in  using  CoPs.  This  was  especially  beneficial  to  those  who  were  not  co¬ 
located  and  did  not  have  the  advantage  of  a  common  network  storage  area,  having  data 
in  one  place  was  complimented  when  the  knowledge  owners  and  administrators  ensured 
that  either  the  relevant  data  was  available  or  links  to  the  data  were  present.  Additionally, 
by  having  the  “authoritative”  document  available  on  the  site,  users  didn’t  have  to  wonder 
if  they  had  the  most  current/relevant  document  available. 

Forums 

This  was  used  primarily  for  posting  comments  or  receiving  feedback.  Because 
anyone  with  access  to  the  CoP  could  read  the  feedback,  the  questions  could  be  answered 
once  rather  than  several  times  as  it  tends  to  be  sent  out  in  e-mail.  Finally,  an  authoritative 
source  could  post  guidance  on  the  forums  and  individuals  could  take  this  as  official 
direction. 
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Connectivity 

Access  to  other  users  was  one  of  the  main  points  applied  to  this  section.  Most 
users  commented  that  there  really  is  not  another  form  of  medium  that  provided  access  to 
others  in  the  manner  that  CoPs  provide.  Furthermore,  use  of  the  AF  Portal  in  allowing 
access  to  users  that  were  on  a  “non-.mil”  domain  was  noted  as  a  positive  step  in 
increasing  connectivity. 

Usage  Issues 

Several  users  commented  on  the  ability  of  the  CoP  to  provide  current  information. 
By  having  a  centralized  repository,  many  users  felt  that  they  did  not  have  to  “reinvent  the 
wheel”  in  regards  to  some  of  the  work,  that  others  may  have  already  done  and  provided 
as  a  template. 

Interface  and  Usability 

CoP’s  compatibility  with  other  programs  currently  loaded  onto  most  desktop 
computers  was  highlighted  as  a  positive  factor  for  this  item.  The  specific  programs 
mentioned  included  the  Microsoft®  Word,  Excel,  and  PowerPoint  software. 

Records  Management 

The  secure  access  provided  to  CoPs  was  noted  as  a  positive  factor  in  using  CoPs. 
One  user  felt  that  because  of  the  closed  access  to  their  CoP,  they  had  control  over  the 
access  to  their  information. 

Training 

Three  users  reported  receiving  an  introductory  e-mail  from  their  administrator  that 
generally  overviewed  their  CoP.  Two  other  users  felt  that  the  system  was  simple  enough 
that  they  did  not  need  training. 
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2b.  What  factors  discouraged  an  individual  from  participating  in  a  particular  CoP 
when  initially  starting  to  use  the  CoP? 

Document  Management 

One  user  commented  that  he  wished  that  there  was  some  form  of  feedback  that 
you  could  apply  to  a  document.  Additionally,  another  user  commented  that  it  was  hard  to 
tell  what  the  authoritative  document  was  on  a  CoP,  so  they  were  leery  of  using  it  as 
formal  guidance. 

Forums 

Posting  feedback  in  forums  was  difficult  for  some  users.  They  felt  that  the 
threads  were  difficult  to  follow  and  this  awkwardness  hindered  communications.  One 
user  noted  the  need  to  post  infonnation  to  the  forum  anonymously.  Another  user  felt  that 
there  needs  to  be  something  more  to  the  user  profiles  than  just  a  name,  office  symbol, 
phone  number,  and  location.  Other  forums  that  he  had  used  in  the  past  provided  a  place 
to  put  more  information  that  he  felt  increased  trust  between  members  who  were 
geographically  separated. 

Connectivity 

Although  the  AF  Portal  was  mentioned  as  a  positive  factor,  some  users  had 
extensive  trouble  accessing  the  CoP  through  the  portal.  In  addition,  one  user  felt  that 
there  were  not  enough  users  on  their  CoP  to  be  productive. 

Usage  Issues 

One  user  felt  that  needed  changes  were  not  made  to  the  CoPs  in  a  timely  manner. 
This  person  was  specifically  talking  about  functional  changes  from  AFKN,  and  not  about 
changes  that  could  be  perfonned  at  the  CoP  administrator’s  level. 
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Interface  and  Usability 

There  were  several  issues  in  this  section,  all  revolving  around  CoPs  at  the  AFKN 
level.  Some  users  felt  that  the  site  was  too  “busy”  and  they  had  trouble  navigating 
around  and  finding  information  as  well  as  uploading  and  viewing  documents.  Others 
wanted  more  customizability  in  the  interface.  One  user  recommended  the  ability  to  have 
a  web  page  located  on  the  CoP  home  page  (hosted  on  AFKN)  to  provide  a  cleaner 
interface  as  well  as  the  ability  to  add  other  general  infonnation  sections  such  as  a  “Hot 
Topics”  section.  Another  suggested  some  form  of  white  board  or  net  meeting 
collaborative  system  to  be  incorporated  into  the  system  so  that  users  could  collaborate 
synchronously. 

Records  Management 

A  few  users  complained  about  how  the  sections  were  broken  down  and  felt  that  a 
structured  system  (taxonomy)  needed  to  be  incorporated  to  help  users  find  the  data  that 
they  wanted.  In  addition,  another  user  felt  that  the  system  did  not  have  enough  pertinent 
data  in  order  to  be  a  good  source  of  information.  Finally,  the  fact  that  there  were  several 
other  system,  to  include  Communities  of  Interest  (Col),  other  CoPs  and  general  web 
pages  out  there  that  provides  different  information.  They  felt  that  there  needed  to  be  a 
single-system  for  this  information. 

Training 

Several  users  commented  that  there  needs  to  be  more  and  diverse  training 
provided  for  users.  These  users  felt  that  they  were  just  given  a  link  on  where  to  go  and 
sign  up,  and  then  it  was  up  to  them  to  figure  it  out. 
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Facilitator 


Some  users  felt  that  there  really  was  not  any  advertising  for  CoPs,  while  others 
felt  that  there  needed  to  be  more  oversight  for  the  CoPs  in  the  form  of  guidance/help  to 
users  and  administrators.  One  user  recommended  AFKN,  or  someone  with  a  reasonable 
background  in  CoPs,  could  provide  feedback  on  a  CoP  to  the  CoP  administrator  on  some 
recurring  basis. 

3a.  What  factors  encouraged  an  individual  to  participate  in  a  particular  CoP  after 
initial  use? 

Document  Management 

Similar  to  users  who  had  used  the  system  for  eight  months  or  less,  posting  and 
sharing  documents,  having  the  data  in  one  place  and  using  the  CoP  as  an  authoritative 
source  came  out  as  positive  factors.  One  user  summed  it  up  by  calling  the  system  a 
“central  repository  for  lessons  learned,  best  practices,  briefings,  guides,  and  templates.” 
Forums 

Several  users  felt  that  using  forums  to  post  comments,  questions,  and  feedback 
was  beneficial  to  them.  One  individual  felt  that  forums  provided  a  place  to  share 
problems  with  others  in  his  community  to  see  if  others  were  seeing  similar  issues. 
Connectivity 

The  ability  to  connect  to  other  users  as  well  as  the  ability  to  find  knowledge 
experts  within  a  specific  area  was  considered  a  positive  factor.  This  was  noted  to  be 
especially  helpful  for  individuals  that  are  geographically  separated. 
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Usage  Issues 

Sharing  information  in  the  form  of  best  practices  and  lessons  learned  was  very 
important.  Additionally,  the  reduction  in  e-mails  due  to  the  information  being  centrally 
located  was  also  highlighted  as  a  positive  factor.  The  users  liked  the  ability  to  go  out  and 
get  infonnation  at  their  convenience  rather  than  waiting  for  it  to  flow  down  through 
“channels.” 

Training 

One  of  the  users  mentioned  the  positive  impact  that  a  personal  introduction  e-mail 
made.  This  e-mail  went  above  the  typical  introductory  e-mail  and  covered  why  they 
should  join  the  CoP 

3b.  What  factors  discouraged  an  individual  from  participating  in  a  particular  CoP 
after  initial  use? 

Document  Management 

Some  users  complained  that  there  were  too  many  lessons  learned  and  best 
practices,  but  not  enough  formal  guidance  on  their  CoP.  In  addition,  users  did  not  have  a 
way  to  know  how  reliable  a  specific  lessons  learned  was. 

Forums 

Several  users  felt  that  the  forums  needed  to  be  more  robust.  They  felt  that  the 
forums  did  not  lend  themselves  to  communication  and  were  hard  to  navigate  and  that 
because  there  is  a  limited  number  of  people  utilizing  it,  that  the  discussions  were 
immaterial.  The  overall  feeling  in  regards  to  forums  was  that  people  just  are  not  using 
them. 
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Connectivity 

One  user  felt  that  the  network  of  people  using  the  system  was  poor  and  attributed 
to  a  lack  of  use.  Additionally,  there  was  a  feeling  that  people  would  forget  about  using 
the  system  and  then  the  information  would  not  be  updated. 

Usage  Issues 

One  user  felt  that  some  of  the  other  members  of  his  CoP  were  afraid  to  use  the 
new  system  due  to  a  fear  of  the  unknown.  Another  user  had  problems  in  the  past  with 
some  of  the  links  to  AFKN  resources  being  broken. 

Interface  and  Usability 

Several  of  the  users  who  had  been  active  with  their  CoP  over  a  year  were  still 
having  problems  navigating  through  the  site,  uploading  and  viewing  documents,  as  well 
as  trouble  logging  onto  the  system.  There  were  also  two  users  noted  the  lack  of 
synchronous  collaboration  capabilities.  Another  user  wanted  to  see  some  kind  of  tie  into 
Outlook’s  calendar  that  would  allow  users  to  update  the  CoP’s  calendar  or  the  CoP’s 
calendar  to  update  or  send  out  meeting  requests  to  users. 

Records  Management 

Users  commented  that  the  records  management  aspect  of  CoPs  was  poor  overall. 
In  addition,  the  need  for  a  taxonomy/hierarchy  was  identified  again.  This  was  a  major 
detractor  cited  by  many  of  the  more  seasoned  users.  One  user  mentioned  that  a  lot  of  the 
administrators/knowledge  owners  did  not  know  when  to  delete  old  or  outdated  material. 
Additionally,  the  users  felt  that  it  was  not  intuitive  where  they  were  supposed  to  place 
their  information. 
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Training 

Need  for  better  introduction  training  was  identified.  Additionally,  several  users 
cited  the  need  for  training  in  general  to  help  utilize  CoPs  better.  Due  to  a  general  lack  of 
awareness,  the  users  felt  that  they  did  not  know  what  a  CoP  could  do  for  them  and 
therefore  couldn’t  take  advantage  of  all  that  the  CoPs  had  to  offer. 

Facilitator 

The  users  felt  that  a  knowledge  champion  could  be  a  focal  point  for  getting  the 
needed  resources  and  recognition  to  those  that  are  using  CoPs.  To  quote  one  user:  “grass 
roots  is  wonderful,  but  to  get  resources  that  you  need  to  make  it  really  go,  you  have  to 
have  some  kind  of  buy  in.”  Several  users  felt  that  a  facilitator  or  knowledge  champion 
would  aid  in  orienting  individuals  who  are  new  to  CoPs.  One  of  the  key  needs  identified 
for  the  implementation  of  a  facilitator  was  the  need  to  “institutionalize”  CoPs. 

4.  How  are  CoP  users  using  CoPs? 

Interview  question  one:  “How  do  you  use  the  system  to  perform  your  job?  ”  was 
primarily  used  to  answer  research  question  4.  There  are  two  main  sections  to  CoPs:  the 
Forum  section  and  the  Document  Management  section.  See  Table  18  for  the  summary  of 
results  on  how  the  CoPs  are  being  used.  The  responses  are  reported  in  three  different 
sections.  First,  if  an  individual  used  forums,  a  “1”  is  placed  in  the  area  used  column 
under  “Forum.”  Similarly,  if  they  used  the  Document  Management  section,  a  “1”  was 
placed  under  “Doc  Mgmt.”  Next,  based  on  the  individuals  response,  a  “1”  was  placed  in 
the  column  that  depicted  how  that  particular  section  was  used.  For  instance,  if  a  person 
used  the  forums  section  to  pass  give  direction/guidance,  a  “1”  was  placed  in  the 
corresponding  column.  In  another  case,  they  may  have  used  the  document  management 
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section  to  post  or  share  lessons  learned.  In  this  case,  a  “1”  was  placed  under  the 
corresponding  column  in  the  document  management  section. 


Area  Used 

Forums 

Document  Management 

Time 
With  CoP 

Forum 

Doc 

Mgmt 

Give/receive 

guidance 

(Directive) 

Post 

Comments/ 

get 

feedback 

Give/receive 

guidance 

(authoritative 

source) 

Post/share 

Documents 

1 

1 

1 

1 

1 

1 

2 

1 

1 

2 

1 

1 

4 

1 

1 

5 

1 

1 

6 

1 

1 

1 

1 

1 

6 

1 

1 

1 

1 

1 

6 

1 

1 

6 

1 

1 

6 

1 

1 

1 

7 

1 

1 

1 

1 

8 

1 

1 

12 

1 

1 

1 

1 

12 

1 

1 

12 

1 

1 

12 

1 

1 

1 

12 

1 

1 

1 

12 

1 

1 

1 

1 

18 

1 

1 

24 

1 

1 

72 

1 

1 

Total  <8 
Months 

5 

11 

3 

4 

4 

9 

Total  >12 
Months 

3 

8 

0 

3 

2 

8 

Total  All 

8 

19 

3 

7 

6 

17 

Table  18  -  Results  of  How  CoPs  are  being  used 


There  were  three  individuals  who  had  used  CoPs  for  eight  months  or  less,  that  use 
forums  to  give/receive  guidance  from  the  forums  section,  whereas  no  one  who  has  been 
using  the  system  for  a  year  or  more  used  forums  for  that  purpose.  Other  than  this 
difference  in  the  forums  usage,  there  were  no  other  significant  differences  in  usage 
between  users  in  the  two  groups. 
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The  interesting  finding  in  regards  to  system  usage  is  the  amount  of  individuals 
that  use  forums  over  those  that  use  the  document  management  section.  As  the  data 
shows,  just  about  everyone  uses  the  document  management  section,  where  less  than  half 
of  the  users  use  the  forums  section. 

Discussion 

In  answering  research  questions  la-le,  several  items  emerged.  First,  all  21 
factors  were  identified  as  significant  in  at  least  one  of  the  five  research  questions.  That  is 
to  say,  that  all  of  the  factors  identified  by  Hinrichsen  and  Fitzgerald  (2004)  had  some 
significant  differences  based  on  the  five  items  that  were  used  to  evaluate  the  data. 
Additionally,  out  of  all  the  factors  evaluated,  Facilitating  Conditions  was  found  to  be 
significant  in  all  research  questions  except  lc;  more  than  any  other  factor.  This  factor  is 
directly  related  to  the  User  Acceptance  Enablers  construct  that  was  found  to  be  very 
predictive  in  the  research  model.  The  factors  of  Anonymity,  Security  Constraints,  and 
Social  Factors  were  the  next  highest  factors  to  stand  out  from  the  evaluations.  Although 
Anonymity  and  Security  Constraints  were  not  directly  evaluated  in  the  current  research’s 
model,  Social  Factors  was  evaluated  and  determined  that  it  was  predictive,  along  with 
User  Acceptance  Enablers. 

Research  question  la 

la.  Based  on  existing  models,  is  there  a  difference  between  factors  based  on 
whether  the  CoP  is  used  by  teams,  function,  or  directorates? 

Overall,  directorates  tend  to  have  a  lower  perception  of  Trust,  Information 
Sharing,  Tolerance,  and  Facilitating  Conditions,  while  having  a  higher  perception  of 
Anonymity  as  opposed  to  teams.  This  inverse  relationship  tends  to  indicate  a  level  of 
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intimacy  (or  expected  intimacy)  that  teams  have.  Functions  are  similar  to  directorates  in 
regards  to  trust  and  Information  Sharing;  this  could  be  associated  to  their  similarity  in 
size.  Functions  are  similar  to  teams  in  regards  to  Facilitating  Conditions  though,  and 
seem  to  relate  to  the  informal  nature  that  teams  and  functions  share.  This  relationship 
will  be  discussed  further  in  the  next  paragraph.  Finally,  there  tended  to  be  a 
disproportionate  amount  of  functional  CoPs  (69%)  compared  to  12%  and  19%  for 
directorates  and  teams  respectively. 

Research  question  lb 

lb.  Based  on  existing  models,  is  there  a  difference  between  factors  based  on 
whether  the  CoP  is  formed  informally  or  formally? 

Surprisingly,  Facilitating  Conditions  emerged  as  the  only  construct  that  was 
affected  by  the  factor  of  Formality.  This  finding  is  counterintuitive  in  that  it  would  be 
expected  that  people  in  a  formally  organized  CoP  would  have  resources  readily  available 
in  the  form  of  training  and  support.  This  finding  demonstrates  the  reliance  on  infonnal 
networks  within  CoPs.  There  was  an  overwhelming  number  of  informal  CoPs  (87%)  as 
opposed  to  only  13%  of  CoPs  being  formally  structured. 

Research  question  lc 

lc.  Based  on  existing  models,  is  there  a  difference  between  factors  based  on 
whether  the  CoP  is  open  or  closed? 

The  finding  that  Security  Constraints  and  Willingness  to  Share  are  being  held 
higher  in  a  closed  CoP  seems  obvious.  This  is  because  the  individuals  who  are  using  a 
closed  CoP  would  tend  to  maintain  a  level  of  protection,  even  at  the  unclassified  level,  of 
their  information;  whereas  if  an  individual  uses  an  open  CoP,  they  would  do  so  with  an 
understanding  that  their  infonnation  was  generally  available  to  anyone.  The  response 
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that  social  factors  are  held  higher  in  open  CoPs  than  in  closed  CoPs  was  unexpected 
though. 

Research  question  Id 

Id.  Based  on  existing  models,  is  there  a  difference  between  factors  based  on  how 
long  the  individual  has  been  with  the  CoP? 

Of  all  the  findings,  this  research  question  is  the  most  intriguing.  The  thing  that 
stands  out  is  the  decrease  in  perceptions  over  time.  This  hints  at  the  need  for  constant  re¬ 
engagement  of  those  who  have  been  with  the  CoP  for  any  length  of  time;  not  only  in 
regards  to  the  items  specifically  identified  as  significant,  but  also  arguably  all  the 
construct  areas.  The  findings  from  this  question  show  the  initial  excitement  of  the  CoPs 
that  gradually  turns  towards  disenchantment  with  the  system  as  time  goes  by.  It  is  also 
important  to  note  the  large  proportion  of  users  who  have  been  using  the  system  for  less 
than  a  year  (74%). 

Anonymity  increasing  over  time  (one  of  the  two  constructs  that  increased)  seems 
to  indicate  a  need  to  not  be  identified  at  times.  It  is  difficult  to  presuppose  what 
organizational  factors  lead  an  individual  to  want  anonymity  as  they  use  the  system  more, 
it  would  be  more  understandable  to  expect  this  to  go  down  as  users  became  more 
comfortable  with  the  system  and  those  who  are  accessing  it.  Security  constraints,  the 
other  construct  that  increased  over  time,  seems  to  indicate  a  need  for  a  more  secure 
environment  in  which  to  share  information.  Although  CoPs  are  relatively  secure,  they 
are  only  cleared  to  handle  infonnation  that  has  been  classified  up  to  “For  Official  Use 
Only.”  Therefore  anything  higher  than  that  (Secret  and  above)  cannot  be  placed  on  the 
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CoP.  At  this  time,  CoPs  are  currently  being  developed  that  AFKN  will  host  on  the  Secret 
Internet  Protocol  Router  Network  (SIPRNET). 

Research  question  le 

le.  Based  on  existing  models,  is  there  a  difference  between  factors  based  on  the 
individual’s  grade/position? 

What  stands  out  in  the  findings  to  this  research  question  is  the  higher  perception 
that  enlisted  users  have  over  officers.  In  general,  enlisted  members  perceive  CoPs  as 
being  more  beneficial  to  them  as  a  whole.  Additionally,  individuals  in  the  higher 
positions  also  have  lower  perception  of  what  CoPs  provide  to  them.  This  was 
particularly  enlightening  in-lieu  of  the  high  amount  (53%)  of  individuals  coded  as  “high.” 
With  the  exception  of  these  differences,  no  trends  seem  to  stand  out  in  this  finding. 
Research  question  2a-3b 

In  discussing  research  questions  2a-3b,  there  were  no  specific  areas  that  stood  out 
between  those  who  had  been  using  the  system  for  less  than  eight  months  and  those  who 
had  been  using  the  system  for  more  than  a  year.  Therefore,  in  an  effort  to  reduce 
duplication,  the  analysis  of  research  questions  2a  and  3a  as  well  as  questions  2b  and  3b 
were  combined. 

2/3a.  What  factors  encouraged  an  individual  to  participate  in  a  particular  CoP? 

The  central  repository  that  CoPs  provide  for  posting  and  sharing  data  was  one  of 
the  biggest  factors  for  using  CoPs.  Currently,  many  users  who  are  co-located  use  shared 
drives  to  store  and  share  infonnation.  Because  of  firewall  or  user  access  limitations,  this 
is  usually  restricted  to  users  within  a  common  domain.  The  CoPs  remove  these 
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geographic  and  administrative  limitations  and  place  responsibility  for  access  to  the 
CoP/knowledge  owners. 

Forums  provide  an  asymmetric  means  for  communication  between  users. 
Currently,  there  is  not  any  other  “official”  means  to  post  messages  and/or  receive 
feedback  in  near  real-time.  Other  systems  that  provide  similar  capabilities  include  web 
pages,  e-mail,  teleconferences,  or  NetMeeting.  The  problem  with  web  pages  is  that  they 
generally  function  in  only  one-way.  E-mail  is  asymmetrical  but  it  is  limited  to  those  that 
are  addressed,  and  the  message  threads  in  the  e-mail  are  sometimes  hard  to  follow  due  to 
the  extraneous  addressing  information  between  the  replies.  Teleconferences  provide  a 
symmetrical  means  for  communicating,  but  users  can  be  overwhelmed  if  there  are  too 
many  people  on  the  call  and  therefore  this  method  of  collaboration  is  limited  in  the 
amount  of  true  participants.  NetMeeting  and  Instant  Messenger  are  good  for  symmetrical 
communication  but  if  nobody  saves  the  transcript  of  the  session,  the  information  shared 
cannot  be  reviewed  by  others. 

2/3b.  What  factors  discouraged  an  individual  from  participating  in  a  particular 

CoP? 

The  main  hindrance  in  using  CoPs  seems  to  be  the  confusing  menus,  and  poor 
customizability  of  the  interface.  Many  of  the  CoPs  desperately  need  some  form  of 
taxonomy  to  assist  users  in  finding  their  needed  information.  This  is  further  complicated 
by  the  inability  of  users  to  see  where  they  are  within  the  organization  structure  of  the  site. 
This  is  especially  problematic  within  the  forums  section. 

Many  users  felt  that  the  CoPs  needed  additional  features  to  make  it  a  truly 
collaborative  tool.  One  area  that  needed  expanding  was  the  forums  section.  This  section 
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limits  the  amount  of  user  information  provided  to  other  users.  Additionally  in  the  case  of 
anonymity,  there  is  too  much  information  provided.  The  overall  perception  in  regards  to 
forums  is  that  they  are  very  rarely  used. 

There  was  also  a  perception  that  there  is  a  lack  of  getting  the  word  out  about  the 
CoPs.  The  users  felt  that  if  others  knew  that  this  resource  was  available  for  them,  that 
there  would  be  more  use  of  the  system.  Although  many  users  commented  that  the  system 
was  really  easy  to  learn,  it  seemed  that  many  of  the  users  were  not  using  CoPs  beyond  a 
basic  level.  Additionally,  the  limited  training  available  is  far  below  what  is  needed. 

Finally,  there  needs  to  be  some  form  of  oversight  for  the  CoPs.  The  need  is  for 
someone  to  go  through  and  organize/archive  data  so  that  the  information  is  usable  and 
current,  while  still  maintaining  a  robust  amount  of  data. 

Research  question  4 

4.  How  are  CoP  users  using  CoPs? 

Based  on  the  results  from  the  interview,  the  main  use  of  CoPs  is  to  provide  a 
place  to  store  data.  Although  some  people  are  using  the  forums  sections  to  do  some 
collaboration. 

Limitations 

The  primary  limitation  of  the  analysis  of  the  previous  research’s  data  is  that  the 
survey  respondents  consisted  of  any  Communities  of  Practice  (CoP)  member  willing  to 
take  the  survey  and  then  “self-report”  their  answers.  Another  potential  limitation  of  the 
survey  is  that  the  instrument  was  not  validated.  Since  the  survey  was  a  combination  of 
two  separate  research  efforts  (Fitzgerald,  2004;  Hinrichsen,  2004),  the  number  of 
questions  used  in  each  study  was  kept  at  a  minimum  to  increase  the  response  rates. 
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Additionally,  all  of  the  survey  respondents  were  members  of  AFKN  CoPs  and  each 
member  of  these  CoPs  support  the  DoD  as  well  as  the  USAF.  Therefore,  the  results  may 
not  be  generalizable  to  CoPs  outside  of  the  military. 

The  primary  limitations  in  regards  to  the  evaluation  of  the  research  model  for  the 
current  research  were  the  low  sample  size  and  response  rate.  Although  there  are 
thousands  of  AFKN  users,  only  2 1  were  interviewed  for  this  research.  Next,  there  is  an 
unknown  amount  of  individuals  that  have  used  CoPs,  and  for  some  reason  or  another, 
they  chose  not  to  continue  using  CoPs.  These  individuals  are  hard  to  identify  and/or 
contact  and  therefore  their  input  is  unavailable.  Additionally,  although  the  questions  are 
validated  based  on  findings  from  previous  TAM  research  (Venkatesh  et  ah,  2003),  there 
may  be  some  dynamic  that  a  knowledge  management  system  introduces  that  is  not 
congruent  with  the  format  of  this  instrument. 

Another  limitation  in  regards  to  the  interview  was  the  distinction  of  new  users  as 
having  used  the  CoP  for  less  than  eight  months  and  more  experienced  users  as  having 
used  the  CoP  for  more  than  a  year.  This  was  an  arbitrary  point  for  delineation,  and  may 
not  accurately  divide  true  “new”  users  from  “experienced”  users. 

Summary 

This  chapter  presented  and  discussed  the  findings  that  were  acquired  during  the 
collection  phase  described  in  Chapter  3.  This  chapter  was  presented  in  four  distinct 
sections.  First,  the  data  from  the  Fitzgerald  &  Hinrichsen  (2004)  studies  was  presented 
with  the  blocking  factors  of  the  CoP’s  functional  organization,  fonnality,  access  (open  or 
closed),  grade/position,  and  length  of  time  using  CoPs.  These  results  were  used  to 
answer  research  questions  la  through  le.  These  research  questions  were  answered  and 
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the  underlying  hypotheses  that  the  five  factors  did  make  a  difference  were  supported. 
Next,  the  results  from  the  pre-interview  survey  were  presented  along  with  the  current 
research  model.  These  results  were  used  to  evaluate  the  research  model  and  to  assist  in 
answering  section  three.  Based  on  the  findings  from  the  pre-interview  survey,  the  current 
research  model  was  refined  to  more  accurately  predict  usage  behavior.  In  the  third 
section,  the  factors  derived  from  the  21  interviews  were  presented.  These  results  were 
used  to  answer  research  questions  2a  through  4.  In  the  fourth  section,  the  findings  from 
the  research  questions  were  discussed  at  length.  Chapter  5  will  provide  conclusions, 
recommendations,  and  possibilities  for  future  research. 
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V.  Recommendations  and  Conclusions 


Overview 

The  purpose  of  this  research  was  to  discover  if  there  are  a  specific  set  of  factors 
that  CoP  or  AFKN  administrators  can  incorporate  into  CoPs  to  encourage  acceptance  and 
use.  In  identifying  these  factors,  it  was  unknown  whether  these  factors  may  or  may  not 
be  affected  based  on  a  CoP’s  functional  makeup,  formality,  access,  length  of  use  or  user’s 
grade.  Based  on  the  findings  presented  in  the  previous  chapter,  there  were  multiple 
factors  that  affected  acceptance  and  use  of  CoPs. 

In  performing  this  study,  the  triangulation  method  that  was  employed  provided 
varying  levels  of  observations.  First,  the  high-level  survey  performed  by  Fitzgerald  and 
Hinrichsen  (2004)  provided  a  broad  view  across  a  large  number  of  CoP  users.  Next,  the 
pre-interview  survey  provided  a  medium-level  view  of  the  issues  and  was  designed  to 
evaluate  technology  acceptance  theory  against  knowledge  management  systems.  Finally, 
the  one-on-one  interviews  provided  a  close-in  view  of  usage  and  sought  out  the  specific 
viewpoint  of  users  in  regards  to  acceptance  and  use  of  AFKN  CoPs. 

Recommendations 

As  stated  in  chapter  one,  the  intention  of  this  research  was  to  use  the  findings  to 
potentially  aid  in  the  modification  and  management  of  existing  AFKN  CoPs,  as  well  as  in 
the  design  and  implementation  of  future  CoPs.  Based  on  this,  the  following  conclusions 
and  recommendations  are  specifically  targeted  towards  AFKN  and/or  CoP  administrators. 
Please  note,  the  following  areas  are  recommendations  and  are  not  intended  to  come 
across  as  saying  that  either  AFKN  or  the  CoP  administrators  are  doing  a  poor  job.  On  the 
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contrary,  this  effort  has  continued  to  grow  and  expand  and  without  the  work  perfonned 
by  these  two  groups,  CoPs  would  have  failed  a  long  time  ago. 

Recommendations  for  AFKN 

Based  on  the  interviews  conducted  for  this  research,  there  are  several  areas  of 
improvement  that  Air  Force  Knowledge  Now  needs  to  engage.  The  recommendations 
have  been  divided  into  two  sections:  Interface  and  Management. 

In  regards  to  the  interface,  many  users  have  stated  that  it  needs  to  be  updated. 
Some  of  specific  comments  were  that  the  site  was  too  busy,  and  not  customizable.  Areas 
that  could  be  targeted  in  the  document  management  section  include  updating  the  menus 
to  clarify  their  meaning.  An  example  of  this  is  the  column  marked  “date”  but  doesn’t 
specify  what  date  it  is  referring  to.  Additionally,  incorporating  a  feedback  mechanism 
(similar  to  that  found  in  the  Deskbook  area  of  AFKN)  would  be  useful. 

The  forums  section  also  needs  a  major  overhaul.  In  general  navigating  CoPs  is 
difficult,  and  this  is  especially  true  in  following  message  threads  in  the  forums, 
furthermore,  users  would  like  to  see  an  expanded  profile  section,  specifically  one  that  is 
linked  to  the  forums.  The  profile  section  could  include  some  further  information  on 
experience,  qualifications,  or  background.  This  would  help  other  users  in  establishing 
some  level  of  trust  and  rapport  with  individuals  that  they  might  have  never  met  in  person. 
AFKN  would  do  well  to  take  the  example  of  commercial  message  boards  and  forums. 
Finally,  AFKN  needs  to  incorporate  other  collaborative  technologies  into  CoPs;  such  as 
some  form  of  White  Board  software  or  a  collaborative  document  editing  system  that 
would  allow  multiple  users  to  work  on  the  same  document  at  the  same  time. 
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From  the  management  point  of  view,  AFKN  administrators  need  to  get  focus  on 
taxonomies  of  the  different  CoPs.  Several  of  those  interviewed  commented  that  there 
seemed  to  be  a  lot  of  duplication  of  effort  in  the  CoPs.  This  is  not  to  say  that  AFKN 
needs  to  take  it  upon  themselves  to  combine  similar  CoPs,  but  they  need  to  devise  a  way 
for  the  common  user  to  be  able  to  sift  through  the  pile  and  find  the  CoP  that  they  need. 
This  could  be  done  in  several  ways.  One  way  would  be  to  have  an  expanded  list  of  CoPs 
that  includes  a  brief  description  of  the  purpose  of  the  CoP.  Another  way  would  be  to 
extend  the  list  of  CoPs  based  on  their  functional  area.  Another  suggestion  would  be  to 
include  a  search  function  that  would  allow  a  user  to  search  the  different  CoP  descriptions 
by  key  word.  Furthennore,  the  use  of  “Neighborhoods”  could  be  expanded  and  links 
back  to  the  neighborhoods  could  be  added  to  CoPs.  These  neighborhoods  could  also 
have  some  form  of  mutual  membership  that  would  provide  membership  regardless  of 
which  specific  CoP  the  user  started  with.  These  suggestions  lead  to  the  next  area  of 
management  that  needs  to  be  recognized:  oversight. 

Although  it  is  not  in  AFKN’s  scope  to  direct  administrators  in  how  to  manage 
their  CoPs,  many  users  interviewed  felt  that  AFKN’s  broad  knowledge  and  experience 
with  CoPs  could  be  extremely  helpful  at  informally  providing  oversight.  Even  if  a  CoP 
administrator  does  not  know  that  they  need  help,  AFKN  personnel  could  routinely  offer 
assistance,  especially  to  CoPs  that  might  be  seeing  a  decrease  in  usage  over  a  set  period 
of  time.  Additionally,  they  could  offer  some  best  practices  to  administrators  to  help  with 
issues  relating  to  CoPs  in  general.  AFKN  has  initially  addressed  this  by  setting  up  the 
KN  Info  Sharing  CoP,  but  they  need  to  be  more  proactive  in  getting  CoP  administrators 
in  to  use  this  CoP. 
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The  last  area  that  AFKN  needs  to  address  is  increasing  awareness.  In  addressing 
awareness,  AFKN  needs  to  get  the  word  out  on  the  functions  and  benefits  of  CoPs.  From 
the  functional  standpoint,  AFKN  needs  to  expand  upon  their  training  and  help  sections. 

As  this  study  has  shown,  people  are  not  being  trained  on  how  the  CoPs  work.  This 
training  should  be  incorporated  into  the  help  system  and  should  take  on  several  different 
forms  such  as  frequently  asked  questions;  computer-based  training;  flash  animations; 
PowerPoint  presentations;  etc. 

Recommendations  for  CoP  Administrators 

Where  AFKN  needs  to  focus  on  CoPs  as  a  whole,  CoP  administrators  need  to 
focus  on  how  their  CoP  differs  from  other  CoPs.  When  referring  to  the  term  CoP 
administrator,  this  title  addresses  several  different  duty  positions  to  include  CoP  owners, 
knowledge  owners,  knowledge  champions,  facilitators  etc.  Although  it  is  beyond  the 
scope  of  this  research  to  define  the  duties  of  these  positions,  they  come  together  to  form 
the  CoP’s  administration  and  their  responsibilities  within  the  CoP  should  be  clearly 
defined  as  to  account  for  the  following  recommendations. 

First  and  foremost,  the  CoP  has  to  have  a  clearly  defined  purpose,  and  this 
purpose  should  be  stated  on  the  front  of  the  CoP  for  all  to  see.  As  AFKN’s  capabilities 
become  more  robust,  this  will  be  a  key  method  to  distinguish  between  similar  CoPs.  One 
user  who  has  had  considerable  experience  with  both  successful  and  unsuccessful  CoPs 
recommended  setting  up  CoPs  around  “products,  services,  and  functions”  as  opposed  to 
being  organizationally  based. 

As  stated  in  the  first  paragraph,  there  are  several  positions  that  are  needed  to  assist 
in  managing  a  CoP.  Some  suggestions  on  how  to  divide  tasks  came  out  of  the  interviews. 
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One  user’s  CoP  divides  the  task  of  administration  with  that  of  the  information  owner.  In 


this  situation,  the  administrator  handles  the  technical  side  of  the  CoP  and  provides 
oversight  to  the  infonnation  owner,  while  the  information  owner  focuses  specifically  on 
the  content  of  the  CoP.  Another  task  that  needs  to  be  performed  is  that  of  the  knowledge 
champion.  This  person  does  not  have  to  be  formally  identified,  or  for  that  matter  have 
any  administrative  responsibilities;  however,  this  person  should  be  in  some  position  of 
influence  (either  formally  or  informally)  to  promote  the  CoP’s  existence  and  use.  In 
addition,  because  this  position  is  not  formally  identified,  the  use  of  several  champions 
proclaiming  a  similar  message  is  very  useful  in  advertising  the  existence  of  the  CoP  to 
potential  users.  The  findings  of  the  research  model  demonstrate  the  need  for  this  form  of 
social  influence  in  the  perceptions  of  the  system’s  ease  of  use  and  perceived  usefulness. 

Based  on  the  findings  from  research  questions  la  through  le,  CoP  administrators 
need  to  identity  the  underlying  organization  of  their  CoP:  is  their  CoP  open  or  closed;  is 
it  formed  around  a  team,  function,  or  directorate;  or  is  the  CoP  going  to  be  required  for 
users  to  perform  their  duties,  or  will  it  be  optional.  These  administrators  also  need  to 
focus  on  their  team’s  rank/position.  In  addition,  as  this  study  showed,  the  length  of  time 
that  a  user  has  been  with  a  CoP  tends  to  have  a  dramatic  effect  on  their  perceptions  of  the 
workings  of  the  CoP. 

As  stated  in  AFKN’s  recommendations,  navigating  within  CoPs  was  identified  as 
being  difficult  and  awkward  by  many  users.  Although  there  is  not  one  specific  taxonomy 
that  will  work  for  every  CoP,  administrators  and  information  owners  need  to  be  cognizant 
of  the  potential  difficulties  in  navigating  the  CoP.  This  area  is  an  example  of  where  a 
separate  CoP  or  AFKN  administrator  could  provide  unbiased  oversight  to  an  information 
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owner.  Next,  CoP  administrators  need  to  provide  links  to  other  similar  CoPs  to  reduce 
the  duplication  of  effort  as  well  as  mutually  support  other  CoP’s  efforts.  AFKN  has 
addressed  this  for  some  CoPs  by  creating  “neighborhoods”  to  group  CoPs  with  similar 
purposes.  The  recommendation  on  this  is,  if  there  is  not  a  neighborhood  that  directly 
relates  to  the  CoP’s  purpose,  CoP  administrators  need  to  either  re-evaluate  the  purpose  of 
their  CoP  or  get  with  AFKN  and  see  if  they  can  create  their  own  neighborhood.  In 
addition,  unlike  in  the  physical  world,  in  the  virtual  world  of  AFKN  a  CoP  could  be  part 
of  several  neighborhoods. 

Finally  and  most  importantly,  CoP  administrators  need  to  perform  an  in-depth 
evaluation  of  their  users  training  and  support.  Of  all  the  key  factors  that  emerged 
between  the  analysis  of  Fitzgerald  and  Hinrichsen’s  (2004)  data,  the  pre-interview 
survey,  and  the  interviews;  Facilitating  Conditions  and  User  Acceptance  Enablers 
emerged  as  being  significant  throughout.  The  CoP  administrators  need  to  be  asking 
themselves  several  questions.  First,  “what  kind  of  training  are  we  providing  and  in  what 
form?”  Just  as  AFKN  needs  to  develop  several  different  methods  of  training,  CoP 
administrators  need  to  be  offering  this  training,  as  well  as  self-developed  training  to  their 
users.  Next,  “are  we  re-engaging  with  people  to  make  sure  that  they  feel  supported?” 
The  findings  from  Fitzgerald  and  Hinrichsen’s  (2004)  data  clearly  showed  that  over  time, 
users  did  not  feel  like  they  were  being  supported.  In  addition,  “how  do  we  expect  our 
users  to  handle  problems?”  There  needs  to  be  clearly  defined  options  for  users  to  seek 
help.  Many  users  preferred  some  form  of  FAQ  or  help  search  function,  but  that  is  not  to 
say  that  everyone  wants  that.  Some  individuals  are  fearful  of  technology,  and  just  want 
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to  contact  someone  either  by  e-mail  or  by  phone.  CoP  administrators  need  to  let  their 
users  know,  to  spell  it  out,  what  they  need  to  do  if  they  run  into  a  problem  with  the  CoP. 

Concurrent  Research 

There  are  currently  two  other  research  projects  studying  Air  Force  Knowledge 
Now  Communities  of  Practice.  Captain  Gary  Felax  perfonned  a  case  study  analysis  on 
the  usability  and  accessibility  of  the  AFKN  web  site  to  be  completed  and  published  in 
March  2005.  Lt.  George  Mendoza  performed  a  content  analysis  of  written  material 
pertaining  to  the  application  of  knowledge  management  (KM)  in  education  searching  for 
what  issues  are  considered  are  considered  key  (most  important).  The  results  of  this 
research  will  fonn  the  foundation  for  the  construction  of  a  KM  model  which  can  be  used 
in  an  actual  academic  setting. 

Suggestions  for  Further  Study 

Based  on  the  small  sample  of  individuals  in  the  pre-interview  survey,  the  results 
although  promising  were  truly  inconclusive.  An  extended  study  based  on  the  Venkatesh 
et  al.  (2003)  instrument  would  prove  useful  in  evaluating  technology  acceptance  in  CoPs 
or  other  collaborative/knowledge  management  support  systems.  Additionally,  a  study  of 
the  impact  of  the  implemented  recommendations  by  both  AFKN  and  CoP  administrators 
would  help  determine  the  extent  that  technology  acceptance  findings  have  on  usage  of 
this  type  system. 

Conclusions 

Although  this  study  was  academic  in  nature,  the  underlying  purpose  was  to 
provide  practitioners  some  direct  guidance  on  how  they  could  provide  a  better  service  to 
their  customers.  Many  similar  studies  in  the  past  had  provided  a  conclusion  that  could 
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not  be  implemented.  As  stated  in  the  limitations  at  the  end  of  Chapter  four;  the 
interviews  that  provided  the  majority  of  the  insight  for  this  study  were  from  a  limited 
sample  of  users  and  might  not  have  actually  identified  the  true  underlying  issues  faced  by 
AFKN  CoP  users.  However,  based  on  the  consistency  of  the  overall  interview  findings, 
coupled  with  the  conclusions  provided  by  the  previous  research  efforts,  the 
recommendations  provided  should  be  of  considerable  use  to  both  Air  Force  Knowledge 
Now  personnel  and  Community  of  Practice  administrators  and  users. 
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Appendix  A  -  Human  Subjects  Review  Board  Approval 


DEPARTMENT  OF  THE  AIR  FORCE 

AIR  FORCE  RESEARCH  LABORATORY  (AFMC) 
WRIGHT-PATTERSON  AIR  FORCE  BASE,  OHIO 


8  September  2004 


MEMORANDUM  FOR  AFIT/ENV 

ATTN:  John  P.  Tate 


FROM:  AFRL/HEH 

SUBJECT:  Approval  for  the  Use  of  Volunteers  in  Demonstrations 

1.  Human  experimentation  as  described  in  Protocol  04-60-E, 

"A  Case  study  of  Technology  Acceptance  in  Online  Communities  of 
Practice",  may  begin. 

2.  In  accordance  with  AFI  40-402,  this  protocol  was  reviewed  and 
approved  by  the  Wright  Site  Institutional  Review  Board  (WSIRB)  on 
30  August  2004,  the  AFRL  Chief  of  Aerospace  Medicine  on  8  September 
2004  . 

3.  Please  notify  the  undersigned  of  any  changes  in  procedures 
prior  to  their  implementation.  A  judgment  will  be  made  at  that 
time  whether  or  not  a  complete  WSIRB  review  is  necessary. 


Signed  8  September  2004 

HELEN  JENNINGS 

Human  Use  Administrator 
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Appendix  B  -  Community  of  Practice  Pre-interview  Survey 


Prior  to  our  interview,  please  fill  out  the  following  demographic  infonnation  and  survey. 
If  you  have  any  questions,  we  can  discuss  them  prior  to  beginning  the  interview. 

Please  be  advised,  all  demographic  information  is  for  statistical  purposes  and  your 
anonymity  will  be  maintained. 

Thank  you, 

Lt  Tate 


Organization: 

Current  duty  description  (brief  description): 

Length  of  time  in  this  position: 

Number  of  CoPs  that  you  are  subscribed/registered  with: 

Name  of  primary  CoP: 

E-mail  address: 

Note:  The  remaining  questions  are  in  regards  to  your  PRIMARY  CoP  only. 

Position  in  your  CoP  (User,  Administrators,  etc.): 

Is  access  to  your  CoP  open  or  closed  (If  you  need  a  password,  it  is  closed): 

How  is  your  CoP  organized;  Team,  function,  directorates,  or  other: 

Is  use  of  the  CoP  mandatory  or  optional: 

How  long  have  you  been  subscribed/registered  with  your  CoP: 

On  a  given  week,  how  many  times  (sessions)  do  you  access  your  CoP: 

How  long  do  you  spend  using  the  system  per  session: 

Underline  all  AF  Knowledge  Now  areas  that  you  use:  CoP  Forums  \  CoP  Document 
Management  \  Search  \  Links  \  Deskbook  \  My  Learning  \  other  (please  specify): 
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Use  the  below  scale  to  answer  questions  1  -  28. 


Please  note  the  term  “system”  refers  to  your  primary  CoP  and  not  other  CoPs, 
features  of  AFKN,  or  the  AF  Portal. _ 


Disagree 

Agree 

Strongly  Quite 

Slightly  Neither  Slightly  Quite  Strongly 

1  2 

3  4  5  6  7 

1  I  find  the  system  useful  in  my  job. 

2  Using  the  system  enables  me  to  accomplish  tasks  more  quickly. 

3  Using  the  system  increases  my  productivity. 

4  If  1  use  the  system,  I  will  increase  my  chances  of  getting  a  raise/promotion. 

5  My  interaction  with  the  system  is  clear  and  understandable. 

6  It  has  been  easy  for  me  to  become  skillful  at  using  the  system. 

7  I  find  the  system  easy  to  use. 

8  Learning  to  operate  the  system  was  easy  for  me. 

9  1  intend  to  use  the  system  in  the  next  <n>  months. 

10  I  predict  I  would  use  the  system  in  the  next  <n>  months. 

11  I  plan  to  use  the  system  in  the  next  <n>  months. 

12  I  have  the  resources  necessary  to  use  the  system. 

13  I  have  the  knowledge  necessary  to  use  the  system. 

14  The  system  is  not  compatible  with  other  systems  I  use. 

15  A  specific  person  (or  group)  is  available  for  assistance  with  system  difficulties. 

16  I  feel  apprehensive  about  using  the  system. 

17  It  scares  me  to  think  that  I  could  lose  a  lot  of  information  using  the  system  by  hitting  the  wrong 
key. 

18  1  hesitate  to  use  the  system  for  fear  of  making  mistakes  I  cannot  correct. 

(For  questions  19-22)  I  could  complete  a  job  or  task  using  the  system. . . 

19  •  If  there  was  no  one  around  to  tell  me  what  to  do  as  I  go. 

20  •  If  I  could  call  someone  for  help  if  I  got  stuck. 

21  •  If  I  had  a  lot  of  time  to  complete  the  job  for  which  the  system  was  provided. 

22  •  If  I  had  just  the  built-in  help  facility  for  assistance. 

23  The  system  is  somewhat  intimidating  to  me. 

24  People  who  influence  my  behavior  think  that  I  should  use  the  system. 

25  My  peers  support  using  the  system 

26  People  who  are  important  to  me  think  that  I  should  use  the  system. 

27  The  senior  management  of  this  business  has  been  helpful  in  the  use  of  the  system. 

28  In  general,  the  organization  has  supported  the  use  of  the  system. 


XX 
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Appendix  C  -  Interview  Questions 


Perceived  Usefulness 

1 .  How  do  you  use  the  system  to  perform  your  job? 

2.  Does  the  system  make  you  more,  less  or  the  same  in  regards  to  productivity? 
Why? 

Ease  of  Use 

3.  Was  the  system  easy  for  you  to  leam?  Why? 

Social  influence 

4a.  How  do  your  colleagues  feel  about  using  the  system? 

4b.  What  about  your  boss? 

User  Acceptance  Enablers 

5 .  What  training  and  or  orientation  did  you  receive  on  the  system? 

Facilitating  conditions 

6a.  What  kind  of  support  system  (help)  do  you  have  for  using  the  system? 

6b.  What  would  you  like  to  have? 

Self-efficacy/Anxiety  (Mix  of  UAE  and  FC) 

7.  How  comfortable  do  you  feel  in  using  the  system?  Why? 

Behavioral  intention  to  use  the  system 

8.  If  given  the  choice  would  you  continue  to  use  the  system?  Why? 

General  Questions 

9.  What  do  you  feel  would  make  the  system  better  at  providing  a  web-based 
collaborative  environment  to  share  information,  conduct  business,  manage  a 
project,  keep  abreast  of  important  group  issues,  or  solve  group  problems? 

10.  Closing  thoughts? 
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Appendix  D  -  Informed  Consent  Release  Form 


Informed  Consent  for  Research  on 
Air  Force  Knowledge  Now  (AFKN)  Communities  of  Practice  (CoP) 

You  are  invited  to  participate  in  a  research  study  of  AFKN  CoPs.  This  research  is  being 
conducted  by  Lt  John  Tate  in  fulfillment  of  a  Masters  Degree  program  at  the  Air  Force  Institute 
of  Technology  (AFIT).  Your  participation  includes  completing  a  28-question  survey  as  well  as  a 
10-question  interview.  The  interview  process  should  last  no  more  than  20  minutes.  If  you  elect 
to  participate  in  the  interview,  you  are  also  consenting  to  have  the  interview  audiotaped  and  to 
being  quoted.  A  copy  of  the  interview  transcripts  will  be  made  available  to  you  for  final  approval 
and  release  prior  to  use. 

PURPOSE: 

The  objective  of  this  research  is  to  conduct  a  case  study  investigation  of  the  acceptance  of  AFKN 
CoPs.  The  findings  from  this  research  will  be  used  to  validate  existing  theories  of  technology 
acceptance  as  well  as  provide  AFKN  and  CoP  administrators  with  insight  on  user  acceptance  in 
order  to  improve  the  efficiency  and  usability  of  the  system. 

PARTICIPATION: 

Your  participation  is  COMPLETELY  VOLUNTARY  however,  your  input  is  important  to  help 
understand  factors  of  acceptance  of  AFKN  CoPs.  Your  name  will  be  protected  in  the  final  write¬ 
up.  You  may  withdraw  from  this  study  at  any  time  without  penalty,  and  your  interview  data  will 
not  be  used  in  the  research.  Your  decision  to  participate  or  withdraw  will  not  jeopardize  your 
relationship  with  your  department,  the  Air  Force  Institute  of  Technology,  the  Air  Force,  or  the 
Department  of  Defense. 

CONFIDENTIALITY: 

ALL  ANSWERS  ARE  STRICTLY  ANONYMOUS.  We  request  demographic  information  in 
order  to  interpret  results  more  accurately  and  to  better  understand  the  factors  of  CoP  acceptance. 
Records  of  your  participation  in  this  study  may  only  be  disclosed  according  to  federal  law, 
including  the  Federal  Privacy  Act,  5  U.S.C.  552a,  and  its  implementing  regulations  (See  Below). 

If  you  have  any  questions  concerning  this  research  and  your  part  in  it,  please  contact  First 
Lieutenant  John  Tate  at  (937)  554-3244  orjohn.tate@afit.edu  or  Dr.  Kevin  Elder  at  (937)  785- 
3636  x4796  or  kevin.elder@afit.edu. 

PARTICIPATION  CONSENT:  YOUR  SIGNATURE  INDICATES  YOUR  WILLINGNESS  TO 
PARTICIPATE. 


Participant’s  Signature/Date  Investigator’s  Signature/Date 

Privacy  Notice: 

In  accordance  with  AFI  37-132,  paragraph  3.2,  the  information  below  is  provided  as  required  by  the  Privacy  Act  of  1974. 

Authority:  10  U.S.C.  8012,  Secretary  of  the  Air  Force;  powers  and  duties;  delegation  by;  implemented  by  AFI  36-2601,  USAF 
Survey  Program. 

Purpose:  To  evaluate  factors  affecting  acceptance  of  Air  Force  communities  of  practice. 

Routine  Use:  To  increase  understanding  of  factors  affecting  acceptance  of  Air  Force  communities  of  practice.  No  analyses  of 
individual  responses  will  be  conducted.  Reports  summarizing  factors  in  CoP  acceptance  may  be  published. 

Disclosure:  Participation  is  VOLUNTARY.  No  adverse  action  will  be  taken  against  any  member  who  does  not  participate  in  this 
survey  or  who  does  not  complete  any  part  of  this  survey  or  interview. 
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Appendix  E  -  Community  of  Practice  Survey 

(Fitzgerald,  2004;  Hinrichsen,  2004), 


Survey  Control  Number:  USAF  SCN  03-112 


PURPOSE: 

Our  research  team  is  investigating  the  effects  of  various  factors  of  use  in  communities  of  practice 
(CoPs)  hosted  at  Air  Force  Knowledge  Now.  Our  goal  is  to  more  fully  understand  factors  that 
promote  and  discourage  CoP  usage.  Results  may  be  beneficial  in  the  future  development  and 
management  of  CoPs. 

PARTICIPATION: 

Your  participation  is  COMPLETELY  VOLUNTARY  however,  your  input  is  important  for  us  to 
understand  factors  of  use  in  Air  Force  CoPs. 

CONFIDENTIALITY: 

ALL  ANSWERS  ARE  STRICTLY  ANONYMOUS.  We  request  demographic  information  in 
order  to  interpret  results  more  accurately  and  to  better  understand  the  factors  of  CoP  usage  being 
researched. 

By  participating  in  this  survey  you  acknowledge  that  you  have  read  the  above  information  and  are 
willing  to  participate  in  the  study. 


Contact  information: 

If  you  have  any  questions  or  comments  about  the  survey,  please  contact  Capt  David  Fitzgerald 
(david.fitzgerald@afit.edu)  or  lLt  Peter  Hinrichsen  (peter.hinrichsen@afit.edu). 


Privacy  Notice: 

In  accordance  with  AFI  37-132,  paragraph  3.2,  the  information  below  is  provided  as  required  by 
the  Privacy  Act  of  1974. 

Authority:  10  U.S.C.  8012,  Secretary  of  the  Air  Force;  powers  and  duties;  delegation  by; 
implemented  by  AFI  36-2601,  USAF  Survey  Program. 

Purpose:  To  evaluate  factors  affecting  usage  within  Air  Force  communities  of  practice. 

Routine  Use:  To  increase  understanding  of  factors  affecting  use  of  Air  Force  communities  of 
practice.  No  analyses  of  individual  responses  will  be  conducted.  Reports  summarizing  factors  in 
CoP  usage  may  be  published. 

Disclosure:  Participation  is  VOLUNTARY.  No  adverse  action  will  be  taken  against  any  member 
who  does  not  participate  in  this  survey  or  who  does  not  complete  any  part  of  this  survey. 
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DEMOGRAPHIC  QUESTIONS  (3  Questions) 

IF  YOU  ARE  A  MEMBER  OF  MORE  THAN  ONE  COMMUNITY  OF  PRACTICE, 
CONSIDER  THE  ONE  YOU  PARTICIPATE  IN  MOST  OFTEN.  ONLY  COMPLETE 
ONE  SURVEY. 


D 1 .  To  which  community  of  practice  do  you  belong?  (List  only  the  community  with 

_ which  you  are  most  involved)  [DROP  DOWN] _ 

D2.  How  many  months  have  you  been  a  member  of  your  CoP?  [DROPDOWN] 

Less  than  1,  1-12,  13-24,  25-36,  more  than  36 _ 

D3.  What  is  your  rank?  [DROP  DOWN]:  E-l  through  E-4,  GS-1  through  GS-5,  E-5  and 
E-6,  GS-6  through  GS-10,  E-7  through  E-9,  GS-1 1  through  GS-1 5,  0-1  through  0-3, 
Contractor,  0-4  through  0-6,  0-7  through  0-10,  Other _ 


FACTORS  AFFECTING  USE  OF  COMMUNITIES  OF  PRACTICE  (43  Questions) 


CAREFULLY  CONSIDER  EACH  STATEMEN' 


USING  THE  BELOW  SCALE: 


Strongly 

Slightly 

Disagree 

Disagree 

Disagree 

Don't  know 

Slightly  Agree 

Agree 

Strongly  Agree 

1 .  Information  obtained  from  my  CoP  is  reliable  enough  to  use  in  my  job. 

2.  Information  is  shared  in  my  CoP. 

3.  If  I  use  my  CoP  I  will  increase  my  chances  of  obtaining  a  promotion. 

4.  I  trust  my  fellow  CoP  members. 

5.  Training  in  the  use  of  my  CoP  was  available  to  me. 

6.  My  CoP  recognizes  or  rewards  its  members  for  making  contributions. 

7.  I  would  participate  more  often  in  my  CoP  if  I  could  remain  anonymous. 

8.  Members  of  my  CoP  are  eager  to  learn  new  things. 

9.  My  supervisor  is  very  supportive  of  my  use  of  CoPs  in  my  job. 

10.  Most  members  of  my  CoP  agree  on  major  issues  discussed  in  our  community. 

1 1 .  The  members  of  my  CoP  are  competent  enough  in  their  job  knowledge  to  provide  accurate  information  to  others  within  the  CoP. 

12.  Members  of  my  CoP  work  to  accomplish  common  goals. 

13.  A  knowledge  champion  is  responsible  for  invigorating  a  CoP,  encouraging  CoP  members  to  participate  and  share  knowledge, 
highlighting  successes,  recognizing  the 

14.  My  fellow  CoP  members  try  new  tools  or  suggestions. 

15.  If  I  use  my  CoP  I  will  increase  my  effectiveness  on  the  job. 

16.  In  order  for  a  CoP  to  thrive,  members  must  understand  that  it  is  okay  to  make  mistakes:  my  fellow  CoP  members  are  patient  with 
people  who  make  honest  mistakes. 

17.  I  have  the  knowledge  necessary  to  use  my  CoP. 

18.  My  CoP  ensures  members  know  where  to  find  resources. 

19.  I  would  share  my  opinions  and  insights  more  often  in  my  CoP  if  I  could  remain  anonymous. 

20.  Teamwork  is  valued  in  my  CoP. 

21 .  The  level  of  security  my  job  deals  with  limits  my  ability  to  use  CoPs  in  my  work. 

22.  My  CoP  encourages  its  members  to  use  materials  originating  outside  our  CoP. 

23.  I  would  participate  more  in  my  CoP  if  the  sharing  of  classified  and  higher  information  were  allowed. 

24.  Members  of  my  CoP  are  technically  competent  enough  to  use  our  CoP. 

25.  In  general,  my  organization  has  supported  my  use  of  CoPs. 

26.  My  CoP  should  rely  on  “tried  and  tested”  tools  to  get  things  done. 

27.  Use  of  CoPs  can  significantly  increase  the  quality  of  output  on  my  job. 

28.  My  community  should  encourage  its  members  to  use  resources  posted  at  our  CoP. 

29.  Use  of  CoPs  will  affect  the  performance  of  my  job. 

30.  Material  originating  outside  my  community  should  not  be  posted  on  my  CoP. 

31.  I  have  no  reservations  about  sharing  my  job  knowledge  with  other  members  of  my  CoP. 

32.  It  is  important  to  be  patient  with  people  who  make  honest  mistakes  in  my  CoP. 

33.  The  efforts  of  my  CoP's  knowledge  owner  affect  how  much  I  participate  within  my  CoP. 

34.  Working  in  teams  is  not  important  in  my  CoP. 

35.  Sharing  my  job  knowledge  with  other  members  of  my  CoP  will  make  me  more  valuable  to  my  organization. 

36.  Members  of  my  community  should  be  highly  proficient  in  using  our  CoP. 

37.  It  is  not  necessary  that  information  be  shared  among  members  of  my  CoP. 
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38.  Members  who  make  contributions  to  my  CoP  should  be  given  credit. 

39.  It  is  not  important  for  CoP  members  to  agree  on  major  issues. 

40.  My  fellow  community  members  should  be  cautious  about  taking  advice  or  using  tools  posted  on  our  CoP. _ 

41.  CoP  members  should  explore  new  or  unfamiliar  areas  of  their  CoP. 

42.  Members  of  my  CoP  should  make  some  concession  to  reach  common  goals. _ 

43.  What  factors,  positive  or  negative,  affect  your  participation  in  your  CoP?  Please  use  the  block  below  to  input  your  comments 
COMMENTS:  [RESPONDENT  WRITE-IN] 


CONCLUSION 


IF  YOU  HAVE  ADDITIONAL  COMMENTS  REGARDING  EXPERIENCES  OR 
OBSERVATIONS  IN  YOUR  CoP  OR  IF  YOU  HAVE  QUESTIONS  REGARDING 
THIS  STUDY,  PLEASE  USE  THE  SPACE  BELOW. 

IF  YOU  WOULD  LIKE  A  RESPONSE  TO  A  COMMENT,  ENTER  YOUR 
CONTACT  INFORMA  TION.  PERSONAL  INFORMA  TION  YOU  PROVIDE  IS 
OPTIONAL  AND  WILL  REMAIN  CONFIDENTIAL. 

COMMENTS:  (250  character  maximum)  [RESPONDENT  WRITE-INI _ 


Survey  Complete 


Thank  you  for  your  participation. 

If  you  would  like  more  information  about  Air  Force  Knowledge  Now,  visit 
https://afkm.wpafb.af.mil/ASPs/cop/Entrv.asp?Filter=00  (from  a  .mil  account) 

If  you  would  like  to  know  more  about  the  Air  Force  Institute  of  Technology,  visit 

http://www.afit.edu/ 
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Appendix  F  -  Compiled  Interview  Findings 

Open-ended  responses  from  individuals  who  had  used  the  CoP  eight  months  or  less. 

Time  with  CoPs:  12  2  4  566666  7  8 


Con 

Item 

4 

9 

13 

16 

17 

1 

2 

3 

5 

8 

21 

20 

Neg 

Pos 

Document  Management 

0 

Need  to  be  able  to  put  feedback  into  documents 

N 

1 

0 

U 

Post/share  documents  and  forms 

P 

P 

P 

P 

P 

P 

P 

P 

0 

8 

u 

Data  in  one  place 

P 

P 

P 

0 

3 

11,1,0 

Use  CoP  as  a  reference  (authoritative  source) 

N 

P 

P 

P 

P 

1 

4 

Forums 

0 

Forums  need  to  be  more  robust 

0 

0 

u.1,0 

Post  comments/questions/feedback  (Forums) 

P 

N 

N 

P 

P 

2 

3 

u 

Answer  a  question  once  rather  than  several  times 

P 

0 

1 

u 

Receive  (official)  guidance 

P 

0 

1 

0 

Like  to  see  team  member's  profiles 

N 

1 

0 

1 

Promotes  interaction 

N 

1 

0 

Connectivity 

U,l 

Connectivity  to  other  users 

P 

P 

P 

P 

0 

4 

u 

Find  knowledge  experts 

0 

0 

EU.SE.U 

Access  to  system 

N 

P 

1 

1 

U 

Enough  users  on  the  CoP 

N 

1 

0 

1 

Use  of  the  AF  Portal 

N 

P 

N 

P 

2 

2 

Usage  Issues 

U 

Don't  have  to  reinvent  the  wheel 

P 

0 

1 

U 

Share  Information  (lessons  learned  etc.) 

0 

0 

U,l 

Gain  knowledge/information 

P 

0 

1 

EU 

Fear  of  the  unknown/making  a  mistake 

0 

0 

0 

Need  to  make  sure  that  the  links  are  working  correctly 

0 

0 

U 

Updates  faster/easier  then  web  pages 

P 

0 

1 

U 

Reduced  e-mail  usage/reliance 

P 

P 

0 

2 

0 

Changes  are  implimented  too  slowly  (AFKN) 

N 

1 

0 

Interface  and  Usability 

0 

AFKN  Needs  to  support  web  pages  (links  to  them) 

N 

1 

0 

EU.SE.1,0 

Trouble  navigating  through  the  site 

N 

N 

N 

N 

4 

0 

0,U 

Need  to  be  able  to  refine  searches 

0 

0 

0 

Site  is  very  busy  (poor  web  interface) 

N 

N 

N 

N 

4 

0 

0 

Need  easier  sign  on 

0 

0 

EU 

Compatible  with  other  programs 

P 

0 

1 

SE,U 

Customizability 

N 

1 

0 

U,EU,SE 

Trouble  uploading/viewing  documents 

N 

1 

0 

0 

Need  white  board/Net  meeting 

N 

N 

N 

3 

0 

Records  Management 

0 

Poor  records  management 

0 

0 

U 

Don't  use  it  all  the  time  due  to  lack  of  data 

N 

1 

0 

0 

Need  to  get  down  to  a  single  system  (CoP,  Col,  etc.) 

N 

1 

0 

0 

Need  to  break  CoPs  into  sections  (hierarchy/taxonomy) 

N 

N 

2 

0 

1 

The  system  is  secure 

P 

0 

1 

Training 

0 

Needs  a  better  introduction  Training  (PowerPoint  etc.) 

N 

N 

2 

0 

UAE 

Felt  that  they  didn't  need  training 

P 

P 

0 

2 

UAE 

Received  intro  e-mail  assistance 

P 

P 

P 

0 

3 

Facilitator  (Knowledge  Champion/CoP  assistance) 

0 

0 

0 

Need  to  set  up  CoPs  around  products/services/functions 

0 

0 

U 

Users  not  from  AFMC/are  unfamiliar  with  CoPs 

0 

0 

0 

Need  a  hierarchy  that  links  CoPs 

0 

0 

0 

Need  knowledge  champion 

0 

0 

0 

Purpose  for  CoP  is  poorly  defined 

0 

0 

0 

Need  to  work  on  advertisement  for  usefulness 

0 

0 

0 

Need  more  oversight 

N 

1 

0 

1 

Advertising  CoPs  in  general 

N 

N 

2 

0 

0 

Hard  to  find  the  specific  CoP  that  you're  looking  for 

N 

N 

2 

0 

0 

Need  AFKN  (or  someone)  to  provide  feedback  on  site 

N 

1 

0 
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Open-ended  responses  from  individuals  who  had  used  the  CoP  12  months  or  more. 

Time  with  CoPs:  12  12  12  12  12  12  18  24  72 


Con 

Item 

10 

11 

12 

14 

18 

19 

15 

7 

6 

Neg 

Pos 

Document  Management 

0 

Need  to  be  able  to  put  feedback  into  documents 

0 

0 

U 

Post/share  documents  and  forms 

P 

P 

P 

0 

3 

u 

Data  in  one  place 

P 

0 

1 

U.1,0 

Use  CoP  as  a  reference  (authoritative  source) 

P 

N 

P 

N 

2 

2 

Forums 

0 

Forums  need  to  be  more  robust 

N 

N 

N 

3 

0 

u,l,0 

Post  comments/questions/feedback  (Forums) 

P 

P 

P 

0 

3 

u 

Answer  a  question  once  rather  than  several  times 

0 

0 

u 

Receive  (official)  guidance 

0 

0 

0 

Like  to  see  team  member's  profiles 

0 

0 

1 

Promotes  interaction 

0 

0 

Connectivity 

U,l 

Connectivity  to  other  users 

P 

P 

0 

2 

u 

Find  knowledge  experts 

P 

0 

1 

EU.SE.U 

Access  to  system 

N 

1 

0 

U 

Enough  users  on  the  CoP 

N 

1 

0 

1 

Use  of  the  AF  Portal 

0 

0 

Usage  Issues 

u 

Don't  have  to  reinvent  the  wheel 

0 

0 

u 

Share  Information  (lessons  learned  etc.) 

P 

P 

P 

P 

P 

0 

5 

U,l 

Gain  knowledge/information 

0 

0 

EU 

Fear  of  the  unknown/making  a  mistake 

N 

1 

0 

0 

Need  to  make  sure  that  the  links  are  working  correctly 

N 

1 

0 

U 

Updates  faster/easier  then  web  pages 

0 

0 

U 

Reduced  e-mail  usage/reliance 

P 

P 

0 

2 

0 

Changes  are  implimented  too  slowly  (AFKN) 

0 

0 

Interface  and  Usability 

0 

AFKN  Needs  to  support  web  pages  (links  to  them) 

0 

0 

EU.SE.1,0 

Trouble  navigating  through  the  site 

N 

1 

0 

0,U 

Need  to  be  able  to  refine  searches 

N 

1 

0 

0 

Site  is  very  busy  (poor  web  interface) 

0 

0 

0 

Need  easier  sign  on 

N 

1 

0 

EU 

Compatible  with  other  programs 

N 

1 

0 

SE,U 

Customizability 

0 

0 

U,EU,SE 

Trouble  uploading/viewing  documents 

N 

1 

0 

0 

Need  white  board/Net  meeting 

N 

N 

2 

0 

Records  Management 

0 

Poor  records  management 

N 

1 

0 

U 

Don't  use  it  all  the  time  due  to  lack  of  data 

0 

0 

0 

Need  to  get  down  to  a  single  system  (CoP,  Col,  etc.) 

0 

0 

0 

Need  to  break  CoPs  into  sections  (hierarchy/taxonomy) 

N 

1 

0 

1 

The  system  is  secure 

0 

0 

Training 

0 

Needs  a  better  introduction  Training  (PowerPoint  etc.) 

N 

1 

0 

UAE 

Felt  that  they  didn't  need  training 

0 

0 

UAE 

Received  intro  e-mail  assistance 

P 

P 

0 

2 

Facilitator  (Knowledge  Champion/CoP  assistance) 

0 

0 

0 

Need  to  set  up  CoPs  around  products/services/functions 

N 

1 

0 

U 

Users  not  from  AFMC/are  unfamiliar  with  CoPs 

N 

1 

0 

0 

Need  a  hierarchy  that  links  CoPs 

N 

N 

2 

0 

0 

Need  knowledge  champion 

N 

N 

2 

0 

0 

Purpose  for  CoP  is  poorly  defined 

N 

N 

N 

3 

0 

0 

Need  to  work  on  advertisement  for  usefulness 

N 

N 

2 

0 

0 

Need  more  oversight 

0 

0 

1 

Advertising  CoPs  in  general 

N 

N 

2 

0 

0 

Hard  to  find  the  specific  CoP  that  you're  looking  for 

0 

0 

0 

Need  AFKN  (or  someone)  to  provide  feedback  on  site 

0 

0 
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